2026-06-09T14:57:57.186052Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:57:57.186094Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:57:57.197624Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:57:57.197741Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:57:57.197929Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:57:57.201778Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:57:57.201839Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-06-09T14:57:57.201847Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T14:57:57.108Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 20 - "100.64.0.3" "python-requests/2.32.5" "fa0ce891-d9d4-4cc5-bfcd-e745a6e3e7c2" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:49922 10.134.0.25:443 100.64.0.3:11448 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T14:57:57.147Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 15 - "100.64.0.2" "python-requests/2.32.5" "6a8ed78e-d693-4411-9788-c527c1baf5cd" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:44054 10.134.0.25:443 100.64.0.2:49480 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T14:57:57.185Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 16 - "100.64.0.4" "python-requests/2.32.5" "4109b226-5e95-4942-acad-27dfddc71aa4" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:49922 10.134.0.25:443 100.64.0.4:10430 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-09T14:57:57.225323Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:57:57.225364Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:57:57.236300Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:57:57.236423Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:57:57.236644Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:57:57.243034Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:57:57.243108Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T14:57:57.243120Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:57:57.275068Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:57:57.275115Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:57:57.285608Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:57:57.285724Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:57:57.289414Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:57:57.289486Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T14:57:57.289498Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:57:57.310986Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:57:57.311034Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:57:57.321725Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:57:57.321851Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:57:57.325307Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:57:57.325365Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-06-09T14:57:57.325376Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:57:57.350408Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:57:57.350451Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:57:57.361576Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:57:57.361690Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:57:57.364978Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:57:57.365037Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T14:57:57.365047Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T14:57:57.224Z] "POST /v1/api-keys/search HTTP/1.1" 200 - via_upstream - "-" 122 332 18 - "100.64.0.3" "python-requests/2.32.5" "f97efdc2-26a8-4f34-bc29-66ab05262435" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:44054 10.134.0.25:443 100.64.0.3:11460 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T14:57:57.274Z] "DELETE /v1/api-keys/dd8adadc-7263-44cf-ae68-d70fc51d246a HTTP/1.1" 200 - via_upstream - "-" 0 289 15 - "100.64.0.4" "python-requests/2.32.5" "06f55f0a-5b0e-4495-ad6d-9f05e3655b86" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:44054 10.134.0.25:443 100.64.0.4:6234 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T14:57:57.310Z] "DELETE /v1/api-keys/34164719-369a-45cc-ae2a-0358bb185017 HTTP/1.1" 200 - via_upstream - "-" 0 289 15 - "100.64.0.2" "python-requests/2.32.5" "1682a3a3-83bd-46d4-b8aa-0f19b87e799f" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:49922 10.134.0.25:443 100.64.0.2:57098 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T14:57:57.349Z] "DELETE /v1/api-keys/2e05a8c0-e747-4ac8-940a-b5fc1239a44b HTTP/1.1" 200 - via_upstream - "-" 0 289 15 - "100.64.0.2" "python-requests/2.32.5" "b03dd1d1-fa20-4cfd-a4b8-b837e79fe7de" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:44054 10.134.0.25:443 100.64.0.2:49486 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-09T14:58:06.433284Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.433333Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.447694Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.447827Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.448054Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.452223Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.452285Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T14:58:06.452296Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.472799Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:58:06.472838Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:58:06.484809Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:58:06.484997Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:58:06.485299Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:58:06.489279Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:58:06.489360Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-06-09T14:58:06.489376Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:58:06.510263Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.510379Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.523282Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.523417Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.523667Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.533273Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.533355Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T14:58:06.533370Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.554012Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:58:06.554053Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:58:06.565837Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:58:06.565957Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:58:06.569779Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:58:06.569836Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-06-09T14:58:06.569846Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:58:06.592144Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:58:06.592180Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:58:06.603318Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:58:06.603422Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:58:06.607224Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:58:06.607281Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-06-09T14:58:06.607291Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T14:58:06.635058Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.635107Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.646967Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.647072Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.647251Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.650867Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.650919Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T14:58:06.650929Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.977505Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.977550Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.991542Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.991674Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.991902Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.994356Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:06.994421Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T14:58:06.994432Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T14:58:06.432Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 68 310 19 - "100.64.0.3" "python-requests/2.32.5" "b7969a5b-0ae5-42dd-97b7-9d9a24eacbcc" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:44054 10.134.0.25:443 100.64.0.3:48426 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T14:58:06.471Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 68 311 17 - "100.64.0.4" "python-requests/2.32.5" "5c494c5d-0a43-49fe-8e74-b50705e4ed5a" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:49922 10.134.0.25:443 100.64.0.4:63282 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T14:58:06.509Z] "POST /v1/api-keys/search HTTP/1.1" 200 - via_upstream - "-" 77 2634 24 - "100.64.0.4" "python-requests/2.32.5" "30611206-3ade-427b-8aee-1147c9890865" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:44054 10.134.0.25:443 100.64.0.4:4406 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T14:58:06.553Z] "DELETE /v1/api-keys/79a7b7b3-fc21-40cb-802f-aeea5f5dcc11 HTTP/1.1" 200 - via_upstream - "-" 0 287 16 - "100.64.0.3" "python-requests/2.32.5" "2612d783-9aa2-43a9-a60e-967a56ac9e15" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:49922 10.134.0.25:443 100.64.0.3:34288 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T14:58:06.591Z] "DELETE /v1/api-keys/3204675f-09d2-4ace-b092-e86d0f0ce211 HTTP/1.1" 200 - via_upstream - "-" 0 287 15 - "100.64.0.2" "python-requests/2.32.5" "d0a0c323-5298-460e-9c36-b437bc587c89" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:49922 10.134.0.25:443 100.64.0.2:40398 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T14:58:06.634Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 30 315 16 - "100.64.0.2" "python-requests/2.32.5" "77daf863-788d-4bd3-b6ea-db5051b89fae" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:44054 10.134.0.25:443 100.64.0.2:47892 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T14:58:06.976Z] "POST /internal/v1/subscriptions/select HTTP/1.1" 200 - via_upstream - "-" 115 330 17 - "100.64.0.3" "python-requests/2.32.5" "81e91155-8cea-40e4-ad34-08e906405f46" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:44054 10.134.0.25:443 100.64.0.3:48428 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-09T14:58:23.653998Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:23.654049Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:23.657414Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:23.657536Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:23.657721Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:23.659599Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T14:58:23.659695Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T14:58:23.659710Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T14:58:23.653Z] "POST /internal/v1/subscriptions/select HTTP/1.1" 200 - via_upstream - "-" 114 91 6 - "100.64.0.4" "python-requests/2.32.5" "bf1043dc-2927-4f08-ad67-413dfce583ab" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:44054 10.134.0.25:443 100.64.0.4:21622 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-09T14:59:32.393916Z info ads ADS: new connection for node:4 2026-06-09T14:59:32.396137Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-06-09T14:59:32.396180Z info cache read certificate from file resource=file-root:system 2026-06-09T14:59:49.084240Z info ads ADS: "" 4 terminated with stream closed [2026-06-09T14:59:55.020Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 0 - "-" "-" "-" "-" "-" - - 10.134.0.25:443 100.64.0.3:12106 - - 2026-06-09T15:00:03.575388Z info ads ADS: new connection for node:5 2026-06-09T15:00:03.578124Z info cache read certificate from file resource=file-root:system 2026-06-09T15:00:20.443277Z info ads ADS: "" 5 terminated with stream closed 2026-06-09T15:00:34.447822Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.447881Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.476595Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.476752Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.496322Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.496408Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T15:00:34.496423Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.524559Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.524609Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.535272Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.535406Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.535783Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.541336Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.541397Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-06-09T15:00:34.541408Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.565812Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.565855Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.581915Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.582065Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.582327Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.582367Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.583892Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.583922Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.584135Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.586075Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.586217Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.586411Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.586942Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.611271Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.611312Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.621660Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.621770Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.621934Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.625087Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.625147Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T15:00:34.625163Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:34.647740Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.647776Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.653657Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.653788Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.654026Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.654057Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.654684Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.654708Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.654908Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.656644Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.656773Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.656940Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.657375Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.678739Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.678772Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.688729Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.688821Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.688972Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.692125Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:34.692177Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-06-09T15:00:34.692187Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T15:00:34.446Z] "DELETE /v1/api-keys/317f0fc2-d722-4e57-81c8-abc3f8843570 HTTP/1.1" 200 - via_upstream - "-" 0 327 49 - "100.64.0.2" "python-requests/2.32.5" "b5c112bf-da47-4b42-83bb-419d49864b4a" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:58816 10.134.0.25:443 100.64.0.2:47492 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T15:00:34.523Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 17 - "100.64.0.2" "python-requests/2.32.5" "5b965c5b-0e17-4843-a3db-d193fdc6ea44" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:58832 10.134.0.25:443 100.64.0.2:10754 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T15:00:34.564Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 283 22 - "100.64.0.3" "python-requests/2.32.5" "fd16e5eb-7c6c-43fc-90bc-8223ede513a7" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.66:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.25:34622 10.134.0.25:443 100.64.0.3:4178 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T15:00:34.610Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 14 - "100.64.0.4" "python-requests/2.32.5" "966e7732-7280-4541-b6fc-1ac2a15394bd" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:58816 10.134.0.25:443 100.64.0.4:32976 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T15:00:34.646Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 281 11 - "100.64.0.4" "-" "0fe69d53-e9e5-4088-9d38-c1d337c3f794" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.66:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.25:34626 10.134.0.25:443 100.64.0.4:33326 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T15:00:34.678Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 317 14 - "100.64.0.2" "python-requests/2.32.5" "3cdcd3a7-5d9f-456a-8eee-f9bf5892d366" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:58832 10.134.0.25:443 100.64.0.2:47498 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-09T15:00:39.723503Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:39.723549Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:39.729012Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:39.729046Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:39.760261Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:39.760304Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:39.770845Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:39.770962Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:39.771168Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:39.774424Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:39.774508Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-06-09T15:00:39.774519Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:39.803697Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:39.803743Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:39.804928Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:39.804965Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T15:00:39.721Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 7 - "100.64.0.3" "python-requests/2.32.5" "2476c485-f76a-489a-8024-bf77c4bdb07f" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.25:443 100.64.0.3:61246 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T15:00:39.759Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 15 - "100.64.0.2" "python-requests/2.32.5" "02675411-9338-4671-a190-2cb73b2d3a6d" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:58832 10.134.0.25:443 100.64.0.2:3750 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T15:00:39.802Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 2 - "100.64.0.3" "python-requests/2.32.5" "43a66ff7-b03e-45d5-8966-7a48e53cf87a" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.25:443 100.64.0.3:4184 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 2026-06-09T15:00:47.113587Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:47.113636Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:47.124994Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:47.125149Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:47.125402Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:47.129237Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:47.129313Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T15:00:47.129333Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:47.168688Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.168734Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.175524Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.175661Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.175928Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.175961Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.177518Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.177544Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.177761Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.180864Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.180997Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.181163Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.181680Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.203601Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.203637Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.211177Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.211307Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.211555Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.211592Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.212131Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.212162Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.212386Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.214577Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.214704Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.214870Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.215366Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.238306Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.238341Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.244543Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.244665Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.244878Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.244906Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.245498Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.245520Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.245707Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.247412Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.247570Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.247734Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:47.248212Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T15:00:47.112Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 16 - "100.64.0.4" "python-requests/2.32.5" "23107a8d-b818-4631-bb65-77443f6a99b0" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:58816 10.134.0.25:443 100.64.0.4:43794 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T15:00:47.166Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 15 - "100.64.0.2" "python-requests/2.32.5" "1ec0c70b-09d0-48df-8cf3-3b11aa5f4b5e" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.66:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.25:34626 10.134.0.25:443 100.64.0.2:61992 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T15:00:47.202Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 13 - "100.64.0.3" "python-requests/2.32.5" "5ac54b1c-f534-45ec-8a8b-12b274c0147b" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.66:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.25:34626 10.134.0.25:443 100.64.0.3:36908 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T15:00:47.237Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 278 11 - "100.64.0.4" "python-requests/2.32.5" "64d494ae-fc1b-4c98-9ffd-8fc9193bf12d" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.66:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.25:34626 10.134.0.25:443 100.64.0.4:43808 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T15:00:47.269328Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:47.269365Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:47.275529Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:47.275662Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:47.275910Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:47.275943Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:47.276547Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:47.276572Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:47.276779Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:47.278557Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:47.278690Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:47.278860Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:47.279310Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T15:00:47.268Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 11 - "100.64.0.4" "python-requests/2.32.5" "cb7b8d94-51c4-4d4c-a39c-7e115cbce24d" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.66:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.25:34622 10.134.0.25:443 100.64.0.4:45490 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T15:00:59.807361Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:59.807409Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:59.823912Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:59.824039Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:59.824256Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:59.837326Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:59.837391Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-06-09T15:00:59.837402Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:59.860750Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:59.860789Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:59.868496Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:59.868628Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:59.868898Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:59.868929Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:59.870305Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:59.870334Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:59.870575Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:59.873294Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:59.873429Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:59.873632Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:59.874195Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:59.898847Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:59.898882Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:59.899874Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:59.899905Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:00:59.924231Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:59.924268Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:59.925172Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:59.925201Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:59.950103Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:59.950143Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:59.954960Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:00:59.954991Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.232164Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.232214Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.242741Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.242870Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.243098Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.246507Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.246567Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-06-09T15:01:00.246578Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T15:00:59.806Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 69 312 30 - "100.64.0.2" "python-requests/2.32.5" "95c6e8c2-6218-47c7-b64e-a4b3c6c5cef7" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:58832 10.134.0.25:443 100.64.0.2:51374 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T15:00:59.858Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 283 15 - "100.64.0.3" "python-requests/2.32.5" "fecbb3e6-2be4-47d8-a6f8-10eb02fd59a2" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.66:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.25:34622 10.134.0.25:443 100.64.0.3:59472 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T15:00:59.897Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 401 - - "-" 66 0 2 - "100.64.0.4" "python-requests/2.32.5" "cdbe079f-7a6c-4542-af2a-e5ed933e06fa" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.25:443 100.64.0.4:25736 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T15:00:59.922Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 401 - - "-" 66 0 2 - "100.64.0.3" "python-requests/2.32.5" "18e5af7f-028a-4673-a6f2-29918af47cec" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.25:443 100.64.0.3:8338 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T15:00:59.948Z] "POST /v1/completions HTTP/1.1" 403 - - "-" 66 98 6 - "100.64.0.2" "python-requests/2.32.5" "6d6f2dd3-5ddd-4859-b17d-0c6ef9596149" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.133.0.34:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.25:443 100.64.0.2:38338 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com llm.premium-simulated-simulated-premium-kserve-route.0 [2026-06-09T15:01:00.231Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 37 331 15 - "100.64.0.2" "python-requests/2.32.5" "9426812b-0c3d-4674-be5f-ee6e6bb44e0f" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:58832 10.134.0.25:443 100.64.0.2:51378 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-09T15:01:00.270928Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:00.270970Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:00.281162Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:00.281274Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:00.285358Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:00.285419Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T15:01:00.285430Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:00.311173Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.311217Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.320813Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.320929Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.321119Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.324376Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.324432Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-06-09T15:01:00.324442Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.345396Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:00.345443Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:00.355602Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:00.355707Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:00.359245Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:00.359300Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T15:01:00.359310Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:00.388354Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:00.388395Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:00.398889Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:00.398995Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:00.399186Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:00.400963Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:00.401016Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T15:01:00.401026Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:00.664954Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.665007Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.671611Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.671761Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.672019Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.672066Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.672681Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.672711Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.672925Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.674805Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.674974Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.675159Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.675654Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:00.700054Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:00.700103Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:00.704667Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:00.704707Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T15:01:00.270Z] "DELETE /v1/api-keys/6a08f4a9-8df2-44f9-ae35-cd61c6e92a31 HTTP/1.1" 200 - via_upstream - "-" 0 307 15 - "100.64.0.3" "python-requests/2.32.5" "f100dfc8-4e5e-4ff2-bfcd-0099bef628a7" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:58816 10.134.0.25:443 100.64.0.3:59488 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T15:01:00.310Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 82 324 14 - "100.64.0.4" "python-requests/2.32.5" "f9a99814-7cd0-442e-a278-bb9a9c789caf" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:58832 10.134.0.25:443 100.64.0.4:25744 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T15:01:00.344Z] "DELETE /v1/api-keys/e842826f-dd47-42d5-8fef-a774e4675952 HTTP/1.1" 200 - via_upstream - "-" 0 301 14 - "100.64.0.2" "python-requests/2.32.5" "fd6b1cd9-ba8b-4248-80a5-aa9b7df47217" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:58816 10.134.0.25:443 100.64.0.2:51392 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T15:01:00.387Z] "POST /v1/api-keys HTTP/1.1" 400 - via_upstream - "-" 114 91 13 - "100.64.0.3" "python-requests/2.32.5" "0f5cac05-f340-4827-b00d-fad5b37ca31e" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:58816 10.134.0.25:443 100.64.0.3:59498 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T15:01:00.663Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 299 12 - "100.64.0.4" "python-requests/2.32.5" "bd9f14f3-09d7-406d-8b92-a2600a8631e4" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.66:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.25:34626 10.134.0.25:443 100.64.0.4:25750 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T15:01:00.698Z] "POST /v1/completions HTTP/1.1" 403 - - "-" 66 98 6 - "100.64.0.2" "python-requests/2.32.5" "80b6611d-4f2b-4431-b455-2aa1d8a07bd6" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.133.0.34:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.25:443 100.64.0.2:51408 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com llm.premium-simulated-simulated-premium-kserve-route.0 2026-06-09T15:01:08.868643Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:08.868689Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:08.874077Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:08.874109Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T15:01:08.867Z] "POST /v1/completions HTTP/1.1" 403 - - "-" 66 98 7 - "100.64.0.3" "python-requests/2.32.5" "d4f6dee1-c567-4afe-9dc2-297263af33dd" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.133.0.34:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.134.0.25:57336 10.134.0.25:443 100.64.0.3:30840 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com llm.premium-simulated-simulated-premium-kserve-route.0 2026-06-09T15:01:33.567230Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:33.567280Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:33.582895Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:33.583032Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:01:33.583229Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T15:01:33.566Z] "POST /maas-api/v1/api-keys HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 87 0 16 - "100.64.0.4" "python-requests/2.32.5" "d7249f93-09c4-4e0e-be92-9c05f094519d" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.134.0.25:443 100.64.0.4:37184 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-09T15:01:58.766409Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:58.766489Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:58.783308Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:58.783425Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:01:58.783613Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T15:01:58.765Z] "POST /maas-api/v1/api-keys HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 93 0 18 - "100.64.0.4" "python-requests/2.32.5" "79872bda-e268-4ac5-8030-446a1f1b6bf8" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.134.0.25:443 100.64.0.4:52080 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-09T15:02:07.449194Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:02:07.449245Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:02:07.457691Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:02:07.457837Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:02:07.458091Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:02:07.458122Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:02:07.459920Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:02:07.459943Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:02:07.460333Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:02:07.474577Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:02:07.474749Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:02:07.474948Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:02:07.475604Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T15:02:07.447Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 28 - "100.64.0.2" "python-requests/2.32.5" "239f5559-fc1b-48d6-9498-a93fd571b6bd" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.66:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.25:34626 10.134.0.25:443 100.64.0.2:62720 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T15:02:24.015540Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.015586Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.031337Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.031494Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.031879Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.039173Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.039238Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T15:02:24.039250Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.063046Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.063083Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.063094Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.069691Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.069804Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.069898Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.070097Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.070124Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.071516Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.071539Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.071549Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.071731Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.071751Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.075477Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.075500Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.075616Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.075637Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.075797Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:24.076237Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T15:02:24.014Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 73 315 24 - "100.64.0.3" "python-requests/2.32.5" "26a8ac34-67a9-4a98-934b-95eaf614f6ce" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.25:48560 10.134.0.25:443 100.64.0.3:40562 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T15:02:24.061Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 14 - "100.64.0.3" "python-requests/2.32.5" "466252e2-49ad-432d-8364-7b18b2935102" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.133.0.34:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.134.0.25:56824 10.134.0.25:443 100.64.0.3:43260 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com llm.premium-simulated-simulated-premium-kserve-route.0 2026-06-09T15:02:48.993720Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:48.993780Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:49.000527Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:49.000702Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:49.000961Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:49.000996Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:49.002781Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:49.002806Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:49.003032Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:49.004817Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:49.004942Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:49.005102Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:02:49.005569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T15:02:48.992Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 290 13 - "100.64.0.4" "python-requests/2.32.5" "bb45a6c2-8adc-45b9-a2c3-f7d04c4319fb" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.66:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.25:34622 10.134.0.25:443 100.64.0.4:12926 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T15:03:05.606118Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:03:05.606165Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:03:05.606177Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:03:05.613361Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:03:05.613519Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:03:05.613758Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:03:05.613789Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:03:05.613804Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:03:05.615395Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:03:05.615419Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:03:05.615517Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:03:05.615655Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:03:05.615729Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:03:05.617576Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:03:05.617665Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:03:05.617720Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:03:05.617883Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:03:05.617917Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:03:05.618488Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T15:03:05.604Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 292 13 - "100.64.0.2" "python-requests/2.32.5" "4bc03a4f-9b67-4d0d-b46c-1fedefb4b12c" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.66:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.25:34622 10.134.0.25:443 100.64.0.2:35524 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T15:03:15.317Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 1988 - "-" "-" "-" "-" "-" - - 10.134.0.25:443 100.64.0.4:12984 - - 2026-06-09T15:03:22.617076Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:03:22.617121Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:03:22.623654Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:03:22.623787Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:03:22.624032Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:03:22.624068Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:03:22.625763Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:03:22.625790Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:03:22.625995Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:03:22.627820Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:03:22.627951Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:03:22.628120Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:03:22.628583Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T15:03:22.615Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 290 13 - "100.64.0.2" "python-requests/2.32.5" "437b75b9-6722-45a2-a761-c362c00ddfad" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.66:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.25:34626 10.134.0.25:443 100.64.0.2:44094 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T15:03:39.643241Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:03:39.643295Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:03:39.648546Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:03:39.648588Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T15:03:39.641Z] "POST /v1/completions HTTP/1.1" 403 - - "-" 66 32 7 - "100.64.0.3" "python-requests/2.32.5" "a1fa9b15-43a0-4d34-8ad3-c227997f5c85" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "10.132.0.66:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.25:34626 10.134.0.25:443 100.64.0.3:47394 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T15:03:48.297305Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:03:48.297358Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:03:48.298783Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:03:48.298812Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T15:03:48.295Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 3 - "100.64.0.4" "python-requests/2.32.5" "924de67e-3879-4373-8ac2-19fe675c521f" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.25:443 100.64.0.4:21930 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 2026-06-09T15:03:56.588985Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:03:56.589037Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:03:56.613352Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:03:56.613530Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T15:03:56.613725Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T15:03:56.588Z] "POST /maas-api/v1/api-keys HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 69 0 25 - "100.64.0.2" "python-requests/2.32.5" "b1a1afcd-3335-447f-b3c3-a001a894c355" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.134.0.25:443 100.64.0.2:53950 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-09T15:04:44.768233Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:04:44.768287Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:04:44.783005Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:04:44.783141Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T15:04:44.783329Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T15:04:44.767Z] "POST /maas-api/v1/api-keys HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 79 0 16 - "100.64.0.3" "python-requests/2.32.5" "8e67b07a-9940-93e3-a931-07770be1d5aa" "maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.134.0.25:443 100.64.0.3:5296 maas.apps.41fbe9e8-3362-4d96-b37e-30060784c6bf.prod.konfluxeaas.com opendatahub.maas-api-route.1