2026-04-21T02:50:31.086254Z info FLAG: --concurrency="0" 2026-04-21T02:50:31.086275Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-04-21T02:50:31.086279Z info FLAG: --help="false" 2026-04-21T02:50:31.086281Z info FLAG: --log_as_json="false" 2026-04-21T02:50:31.086284Z info FLAG: --log_caller="" 2026-04-21T02:50:31.086286Z info FLAG: --log_output_level="default:info" 2026-04-21T02:50:31.086288Z info FLAG: --log_stacktrace_level="default:none" 2026-04-21T02:50:31.086293Z info FLAG: --log_target="[stdout]" 2026-04-21T02:50:31.086297Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-04-21T02:50:31.086299Z info FLAG: --outlierLogPath="" 2026-04-21T02:50:31.086301Z info FLAG: --profiling="true" 2026-04-21T02:50:31.086303Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-04-21T02:50:31.086305Z info FLAG: --proxyLogLevel="warning" 2026-04-21T02:50:31.086307Z info FLAG: --serviceCluster="istio-proxy" 2026-04-21T02:50:31.086309Z info FLAG: --stsPort="0" 2026-04-21T02:50:31.086311Z info FLAG: --templateFile="" 2026-04-21T02:50:31.086313Z info FLAG: --tokenManagerPlugin="" 2026-04-21T02:50:31.086316Z info FLAG: --vklog="0" 2026-04-21T02:50:31.086318Z info Version 1.26.2-4b1e497cd1c6b51e6a45e6c0abe9aab6bb19e0a8-Clean 2026-04-21T02:50:31.086323Z info Set max file descriptors (ulimit -n) to: 1048576 2026-04-21T02:50:31.086466Z info Proxy role ips=[10.132.0.24] type=router id=maas-default-gateway-openshift-default-845c6b4b48-m5n8z.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-04-21T02:50:31.086511Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-04-21T02:50:31.087810Z info cpu limit detected as 2, setting concurrency 2026-04-21T02:50:31.088055Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-04-21T02:50:31.088066Z info JWT policy is third-party-jwt 2026-04-21T02:50:31.088069Z info using credential fetcher of JWT type in cluster.local trust domain 2026-04-21T02:50:31.090846Z info Starting default Istio SDS Server 2026-04-21T02:50:31.090872Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-04-21T02:50:31.090896Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-04-21T02:50:31.091163Z info Opening status port 15020 2026-04-21T02:50:31.092061Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-04-21T02:50:31.092737Z info sds Starting SDS grpc server 2026-04-21T02:50:31.092755Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-04-21T02:50:31.093799Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-04-21T02:50:31.094846Z info Starting proxy agent 2026-04-21T02:50:31.094883Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-04-21T02:50:31.149637Z warning envoy main external/envoy/source/server/server.cc:874 Usage of the deprecated runtime key overload.global_downstream_max_connections, consider switching to `envoy.resource_monitors.global_downstream_max_connections` instead.This runtime key will be removed in future. thread=8 2026-04-21T02:50:31.150286Z warning envoy main external/envoy/source/server/server.cc:970 There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor. thread=8 2026-04-21T02:50:31.156055Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-04-21T02:50:31.224393Z info cache generated new workload certificate resourceName=default latency=131.550352ms ttl=23h59m59.775610923s 2026-04-21T02:50:31.224427Z info cache Root cert has changed, start rotating root cert 2026-04-21T02:50:31.224461Z info cache returned workload trust anchor from cache ttl=23h59m59.775539321s 2026-04-21T02:50:31.226284Z info ads ADS: new connection for node:1 2026-04-21T02:50:31.226330Z info cache returned workload certificate from cache ttl=23h59m59.773671238s 2026-04-21T02:50:31.226929Z info ads ADS: new connection for node:2 2026-04-21T02:50:31.226989Z info cache returned workload trust anchor from cache ttl=23h59m59.773012217s 2026-04-21T02:50:32.126209Z info Readiness succeeded in 1.042437349s 2026-04-21T02:50:32.126502Z info Envoy proxy is ready 2026-04-21T02:52:16.668476Z info wasm fetching image kuadrant/wasm-shim from registry quay.io with tag v0.12.1 2026-04-21T02:53:30.531185Z info ads ADS: new connection for node:3 2026-04-21T02:53:30.531458Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-04-21T02:53:30.531545Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-04-21T02:55:06.240680Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 [2026-04-21T02:55:06.207Z] "POST /v1/api-keys HTTP/2" 201 - via_upstream - "-" 55 321 33 - "100.64.0.3" "curl/7.76.1" "0c32cbe6-d401-45e8-9f53-6d1dc44a0bcb" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.134.0.58:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.24:52700 10.132.0.24:443 100.64.0.3:15562 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-04-21T02:55:06.351827Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-04-21T02:55:06.354166Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 [2026-04-21T02:55:06.331Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 20 - "100.64.0.4" "Go-http-client/1.1" "9a9b593d-9427-46d7-b473-b9e5e2e13847" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.24:49392 10.132.0.24:443 100.64.0.4:4320 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-04-21T02:55:06.288Z] "GET /v1/models HTTP/2" 200 - via_upstream - "-" 0 590 66 - "100.64.0.2" "curl/7.76.1" "d94ac06f-3c84-424c-b31f-6220d42fead8" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.134.0.58:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.24:52700 10.132.0.24:443 100.64.0.2:57008 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-04-21T02:55:06.420Z] "POST /v1/chat/completions HTTP/2" 200 - via_upstream - "-" 100 316 15 - "100.64.0.2" "curl/7.76.1" "81bc396a-f339-429f-a921-7b822d8e968a" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.24:49406 10.132.0.24:443 100.64.0.2:42624 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 2026-04-21T02:55:06.888869Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 [2026-04-21T02:55:06.474Z] "POST /v1/chat/completions HTTP/2" 200 - via_upstream - "-" 100 469 9 - "100.64.0.3" "curl/7.76.1" "df9b29f9-ffa7-48f9-8638-66344b8a79bd" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.24:49406 10.132.0.24:443 100.64.0.3:26964 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-04-21T02:55:06.513Z] "POST /v1/chat/completions HTTP/2" 200 - via_upstream - "-" 100 365 9 - "100.64.0.3" "curl/7.76.1" "89f57672-3a6c-43f5-a2bf-9199b3ae2368" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.24:49406 10.132.0.24:443 100.64.0.3:15572 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-04-21T02:55:06.555Z] "POST /v1/chat/completions HTTP/2" 200 - via_upstream - "-" 100 359 10 - "100.64.0.4" "curl/7.76.1" "3197e3d3-99d1-48a3-8164-588a6d00734c" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.24:49406 10.132.0.24:443 100.64.0.4:24572 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-04-21T02:55:06.589Z] "POST /v1/chat/completions HTTP/2" 200 - via_upstream - "-" 100 352 11 - "100.64.0.2" "curl/7.76.1" "f8efd2d1-0826-4f31-9c42-f78eb48372e8" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.24:49392 10.132.0.24:443 100.64.0.2:57010 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-04-21T02:55:06.624Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 19 - "100.64.0.4" "curl/7.76.1" "97e0ffe9-f589-4302-856e-5d3f7f32f31f" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.24:49392 10.132.0.24:443 100.64.0.4:4322 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-04-21T02:55:06.669Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 8 - "100.64.0.3" "curl/7.76.1" "f822926a-63e1-4ab9-9d94-bacf01e66241" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.24:443 100.64.0.3:15584 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-04-21T02:55:06.711Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 7 - "100.64.0.2" "curl/7.76.1" "ae04ba7e-8243-4cef-bb84-589d48d14c25" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.24:49422 10.132.0.24:443 100.64.0.2:57014 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-04-21T02:55:06.746Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 8 - "100.64.0.4" "curl/7.76.1" "3cad9b81-96c1-41ca-aadb-22ebf2ce1f8b" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.24:443 100.64.0.4:4334 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-04-21T02:55:06.778Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 7 - "100.64.0.3" "curl/7.76.1" "93109e39-d38a-4d11-ad4a-807ced063bc0" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.24:49406 10.132.0.24:443 100.64.0.3:15592 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-04-21T02:55:06.812Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 8 - "100.64.0.2" "curl/7.76.1" "a1fc52d0-a7b2-4487-b230-4ac51580465a" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.24:443 100.64.0.2:42634 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-04-21T02:55:06.846Z] "POST /llm/facebook-opt-125m-simulated/v1/chat/completions HTTP/2" 401 - - "-" 100 0 1 - "100.64.0.2" "curl/7.76.1" "c669895f-055f-4ad2-b4f8-91f0a55f0653" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.24:443 100.64.0.2:57024 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-04-21T02:55:06.878Z] "DELETE /v1/api-keys/f8f95de4-d951-4239-aa9a-86a9862d87b2 HTTP/2" 200 - via_upstream - "-" 0 334 10 - "100.64.0.3" "curl/7.76.1" "cab3e65f-039f-491f-8798-dda78a4e7519" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.134.0.58:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.24:52700 10.132.0.24:443 100.64.0.3:26972 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-04-21T02:55:39.141776Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-04-21T02:55:39.193506Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-04-21T02:55:39.194692Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.194758Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.194795Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-04-21T02:55:39.194805Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.243898Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.243940Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.250314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.250452Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.250757Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.250788Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.252081Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.252109Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.283912Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.283954Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.290192Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.290327Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.290616Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.290650Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.291030Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.291052Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.377038Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.377080Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.383368Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.383515Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.383818Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.383851Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.384241Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.384265Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.408332Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.408374Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.414239Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.414376Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.414654Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.414684Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.415049Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.415071Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.438561Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.438598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.444398Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.444568Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.444843Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.444874Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.445186Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.445208Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-04-21T02:55:39.130Z] "POST /v1/api-keys HTTP/2" 201 - via_upstream - "-" 55 321 11 - "100.64.0.4" "curl/7.76.1" "dcb6ca21-9ecd-4627-9eff-e5f0f74f12d5" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.134.0.58:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.24:52700 10.132.0.24:443 100.64.0.4:64236 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-04-21T02:55:39.186Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 7 - "100.64.0.4" "Go-http-client/1.1" "5e594b0f-f6f3-45e9-b05e-680df1eb9b7f" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.24:49426 10.132.0.24:443 100.64.0.4:4320 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-04-21T02:55:39.180Z] "GET /v1/models HTTP/2" 200 - via_upstream - "-" 0 590 14 - "100.64.0.3" "curl/7.76.1" "745efdb2-7242-4e35-9dcb-10b25defd40a" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.134.0.58:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.24:52700 10.132.0.24:443 100.64.0.3:27986 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-04-21T02:55:39.243Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 9 - "100.64.0.4" "curl/7.76.1" "0e2e6940-5569-4a41-9ced-4cf410df1489" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.24:49426 10.132.0.24:443 100.64.0.4:27180 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-04-21T02:55:39.283Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 8 - "100.64.0.2" "curl/7.76.1" "9407b151-d135-469c-8df2-d4c5000de3c1" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.24:443 100.64.0.2:24224 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-04-21T02:55:39.314Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 7 - "100.64.0.4" "curl/7.76.1" "8451117c-e6ce-4567-99ea-431cd35440a2" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.24:49442 10.132.0.24:443 100.64.0.4:64244 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-04-21T02:55:39.345Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 7 - "100.64.0.3" "curl/7.76.1" "195a37da-c355-4161-a764-ec69e6d23d6e" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.24:443 100.64.0.3:27994 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-04-21T02:55:39.376Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 8 - "100.64.0.2" "curl/7.76.1" "29aa859b-4ade-46c2-8acb-60286409514c" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.24:37688 10.132.0.24:443 100.64.0.2:45034 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-04-21T02:55:39.407Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 7 - "100.64.0.2" "curl/7.76.1" "eaa94935-a58d-471b-b208-bb4179317bae" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.24:443 100.64.0.2:24238 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-04-21T02:55:39.437Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 7 - "100.64.0.3" "curl/7.76.1" "08f3f0fa-73b8-4f97-83a3-516e79190c98" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.24:37710 10.132.0.24:443 100.64.0.3:3872 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 2026-04-21T02:55:39.503501Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-04-21T02:55:39.503573Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-04-21T02:55:39.503873Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-04-21T02:55:39.503904Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-04-21T02:55:39.525895Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.525930Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.532368Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.532498Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.532767Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.532797Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.533107Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.533129Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.556563Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-04-21T02:55:39.556602Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-04-21T02:55:39.562445Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-04-21T02:55:39.562608Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-04-21T02:55:39.562851Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-04-21T02:55:39.562889Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-04-21T02:55:39.563196Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-04-21T02:55:39.563224Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-04-21T02:55:39.585929Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.585963Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.586851Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.586880Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-04-21T02:55:39.615823Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-04-21T02:55:39.615865Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-04-21T02:55:39.621598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-04-21T02:55:39.621710Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-04-21T02:55:39.626362Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-04-21T02:55:39.626429Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-04-21T02:55:39.626481Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-04-21T02:55:39.626495Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-04-21T02:55:39.467Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 7 - "100.64.0.4" "curl/7.76.1" "acdda386-3641-4cd6-a275-2e79e9b90db5" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.24:37704 10.132.0.24:443 100.64.0.4:64254 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-04-21T02:55:39.496Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 7 - "100.64.0.4" "curl/7.76.1" "b984ed20-00df-424d-8a32-8d9032309a32" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.24:443 100.64.0.4:27184 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-04-21T02:55:39.525Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 8 - "100.64.0.2" "curl/7.76.1" "0c30da76-4921-4320-857b-dbc1453b286e" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.24:443 100.64.0.2:45044 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-04-21T02:55:39.556Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 7 - "100.64.0.3" "curl/7.76.1" "686d12db-0b96-4705-ae11-9b480967cbbb" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.132.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.24:37726 10.132.0.24:443 100.64.0.3:3874 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-04-21T02:55:39.585Z] "POST /llm/facebook-opt-125m-simulated/v1/chat/completions HTTP/2" 401 - - "-" 100 0 1 - "100.64.0.4" "curl/7.76.1" "1e927b15-bc2c-419d-9c8b-8568e876d0e5" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.24:443 100.64.0.4:27188 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-04-21T02:55:39.614Z] "DELETE /v1/api-keys/e04d93da-94ea-4b39-9ee9-1d862a326738 HTTP/2" 200 - via_upstream - "-" 0 334 11 - "100.64.0.3" "curl/7.76.1" "d6706a7a-700f-4760-ac8a-6bd04ab71e1d" "maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com" "10.134.0.58:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.24:46816 10.132.0.24:443 100.64.0.3:28004 maas.apps.ed445c61-d152-4989-9015-8a2641264289.prod.konfluxeaas.com opendatahub.maas-api-route.1