2026-06-08T21:20:56.266193Z info FLAG: --concurrency="0" 2026-06-08T21:20:56.266221Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-06-08T21:20:56.266225Z info FLAG: --help="false" 2026-06-08T21:20:56.266228Z info FLAG: --log_as_json="false" 2026-06-08T21:20:56.266230Z info FLAG: --log_caller="" 2026-06-08T21:20:56.266232Z info FLAG: --log_output_level="default:info" 2026-06-08T21:20:56.266234Z info FLAG: --log_stacktrace_level="default:none" 2026-06-08T21:20:56.266293Z info FLAG: --log_target="[stdout]" 2026-06-08T21:20:56.266296Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-06-08T21:20:56.266298Z info FLAG: --outlierLogPath="" 2026-06-08T21:20:56.266300Z info FLAG: --profiling="true" 2026-06-08T21:20:56.266302Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-06-08T21:20:56.266304Z info FLAG: --proxyLogLevel="warning" 2026-06-08T21:20:56.266306Z info FLAG: --serviceCluster="istio-proxy" 2026-06-08T21:20:56.266308Z info FLAG: --stsPort="0" 2026-06-08T21:20:56.266310Z info FLAG: --templateFile="" 2026-06-08T21:20:56.266312Z info FLAG: --tokenManagerPlugin="" 2026-06-08T21:20:56.266315Z info FLAG: --vklog="0" 2026-06-08T21:20:56.266317Z info Version 1.27.3-f6a985379fd8fe1e5362851fe5c24e3730f643f9-Clean 2026-06-08T21:20:56.266472Z info Proxy role ips=[10.134.0.49] type=router id=maas-default-gateway-openshift-default-687ff6996-m8vb8.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-06-08T21:20:56.266519Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-06-08T21:20:56.267982Z info cpu limit detected as 2, setting concurrency 2026-06-08T21:20:56.268653Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-06-08T21:20:56.268669Z info JWT policy is third-party-jwt 2026-06-08T21:20:56.268673Z info using credential fetcher of JWT type in cluster.local trust domain 2026-06-08T21:20:56.270956Z info Opening status port 15020 2026-06-08T21:20:56.270969Z info Starting default Istio SDS Server 2026-06-08T21:20:56.270990Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-06-08T21:20:56.271007Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-06-08T21:20:56.271957Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-06-08T21:20:56.272895Z info sds Starting SDS grpc server 2026-06-08T21:20:56.272917Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-06-08T21:20:56.273882Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-06-08T21:20:56.275082Z info Starting proxy agent 2026-06-08T21:20:56.275123Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-06-08T21:20:56.332845Z warning envoy main external/envoy/source/server/server.cc:901 Usage of the deprecated runtime key overload.global_downstream_max_connections, consider switching to `envoy.resource_monitors.global_downstream_max_connections` instead.This runtime key will be removed in future. thread=9 2026-06-08T21:20:56.333168Z warning envoy main external/envoy/source/server/server.cc:997 There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor. thread=9 2026-06-08T21:20:56.339676Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-06-08T21:20:56.429651Z info ads ADS: new connection for node:1 2026-06-08T21:20:56.430145Z info ads ADS: new connection for node:2 2026-06-08T21:20:56.882812Z info cache generated new workload certificate resourceName=default latency=609.79622ms ttl=23h59m59.117193262s 2026-06-08T21:20:56.882889Z info cache Root cert has changed, start rotating root cert 2026-06-08T21:20:56.882947Z info cache returned workload trust anchor from cache ttl=23h59m59.117053563s 2026-06-08T21:20:56.882974Z info cache returned workload certificate from cache ttl=23h59m59.117031023s 2026-06-08T21:20:56.883042Z info cache returned workload trust anchor from cache ttl=23h59m59.116959374s 2026-06-08T21:20:56.883405Z info cache returned workload trust anchor from cache ttl=23h59m59.116596223s 2026-06-08T21:20:56.930568Z info wasm fetching image kuadrant/wasm-shim from registry quay.io with tag v0.12.1 2026-06-08T21:21:01.541155Z info Readiness succeeded in 5.277435063s 2026-06-08T21:21:01.541431Z info Envoy proxy is ready [2026-06-08T21:21:37.941Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 0 - "-" "-" "-" "-" "-" - - 10.134.0.49:443 100.64.0.4:22082 44.223.141.229 - 2026-06-08T21:22:00.025911Z info ads ADS: new connection for node:3 2026-06-08T21:22:00.026135Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-06-08T21:22:00.026168Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-06-08T21:24:41.884103Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-08T21:24:41.983960Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-08T21:24:41.986212Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 [2026-06-08T21:24:41.794Z] "POST /v1/api-keys HTTP/2" 201 - via_upstream - "-" 55 320 89 - "100.64.0.3" "curl/7.76.1" "19e1439d-9e43-41ab-9e96-8a66840160ed" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.134.0.56:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.49:49066 10.134.0.49:443 100.64.0.3:3042 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T21:24:41.956Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 27 - "10.134.0.56" "Go-http-client/1.1" "106308b9-5e36-4382-8340-6d05a5f582fb" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.133.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.49:59816 10.134.0.49:443 10.134.0.56:40816 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-08T21:24:41.932Z] "GET /v1/models HTTP/2" 200 - via_upstream - "-" 0 590 53 - "100.64.0.2" "curl/7.76.1" "267967f8-8e33-40d7-863c-dd94ac328fc2" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.134.0.56:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.49:49066 10.134.0.49:443 100.64.0.2:45784 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T21:24:42.060Z] "POST /v1/chat/completions HTTP/2" 200 - via_upstream - "-" 100 512 16 - "100.64.0.4" "curl/7.76.1" "0c4157f5-e357-4088-a615-da91a5c5c133" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.133.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.49:59818 10.134.0.49:443 100.64.0.4:55102 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-06-08T21:24:42.113Z] "POST /v1/chat/completions HTTP/2" 200 - via_upstream - "-" 100 351 12 - "100.64.0.4" "curl/7.76.1" "aed79121-60fb-4959-bed0-ecd7315fd1e1" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.133.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.49:59816 10.134.0.49:443 100.64.0.4:47144 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-06-08T21:24:42.163Z] "POST /v1/chat/completions HTTP/2" 200 - via_upstream - "-" 100 384 11 - "100.64.0.3" "curl/7.76.1" "59c7a609-d9f9-4eda-abaf-cc7fb7a9eb49" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.133.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.49:59816 10.134.0.49:443 100.64.0.3:3056 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-06-08T21:24:42.210Z] "POST /v1/chat/completions HTTP/2" 200 - via_upstream - "-" 100 516 12 - "100.64.0.2" "curl/7.76.1" "3789ce8f-67d3-46b3-b484-c86aae7411f2" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.133.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.49:59818 10.134.0.49:443 100.64.0.2:63054 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-06-08T21:24:42.247Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 8 - "100.64.0.2" "curl/7.76.1" "d1468465-00db-4b21-b633-4cd0758299e0" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.133.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.49:59818 10.134.0.49:443 100.64.0.2:45794 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-06-08T21:24:42.292Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 9 - "100.64.0.3" "curl/7.76.1" "390f78e9-20e5-4efe-8482-f1d282453b44" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.133.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.49:443 100.64.0.3:54880 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-06-08T21:24:42.325Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 8 - "100.64.0.4" "curl/7.76.1" "92cd0484-1471-4d20-a214-ff584d30110b" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.133.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.49:59832 10.134.0.49:443 100.64.0.4:55108 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-06-08T21:24:42.384Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 9 - "100.64.0.4" "curl/7.76.1" "7f626420-250a-4373-9d20-a40c55a68e81" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.133.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.49:443 100.64.0.4:47154 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-06-08T21:24:42.418Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 9 - "100.64.0.2" "curl/7.76.1" "8eed88a2-b653-4831-bf08-7ace1288563a" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.133.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.49:59842 10.134.0.49:443 100.64.0.2:63060 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-06-08T21:24:42.473Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 9 - "100.64.0.3" "curl/7.76.1" "f7444482-fd2d-462b-aaa7-f03f7e0ee11b" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.133.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.49:443 100.64.0.3:3064 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-06-08T21:24:42.530Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 8 - "100.64.0.2" "curl/7.76.1" "8755e697-cfdb-4267-8a68-a18cd1525ab9" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.133.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.49:59856 10.134.0.49:443 100.64.0.2:45798 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 2026-06-08T21:24:42.599361Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:42.610342Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:42.610541Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:42.621395Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:42.621488Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-08T21:24:42.621501Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-08T21:24:42.567Z] "POST /llm/facebook-opt-125m-simulated/v1/chat/completions HTTP/2" 401 - - "-" 100 0 2 - "100.64.0.3" "curl/7.76.1" "1a9b6a2b-a6ef-40c1-a84d-ccaebd7c9ee8" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.49:443 100.64.0.3:54890 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-06-08T21:24:42.598Z] "DELETE /v1/api-keys/7545a7fd-a60e-421d-8bdb-e4b0aa0df5ed HTTP/2" 200 - via_upstream - "-" 0 334 23 - "100.64.0.4" "curl/7.76.1" "95d28e92-5333-437e-8a8c-bad24e7835f1" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.134.0.56:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.49:49066 10.134.0.49:443 100.64.0.4:55112 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 2026-06-08T21:24:49.529151Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-08T21:24:49.556436Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:49.556479Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-08T21:24:49.525Z] "GET /health HTTP/2" 200 - via_upstream - "-" 0 20 4 - "100.64.0.4" "curl/7.76.1" "dd1821ee-53d6-4b05-b9d3-947fe6101bbf" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.134.0.56:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.49:60472 10.134.0.49:443 100.64.0.4:17058 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 2026-06-08T21:24:49.567925Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:49.568093Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:49.569975Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:49.570082Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-08T21:24:49.570095Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:50.032049Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:50.032094Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:50.034717Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:50.034833Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:50.035041Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:50.038870Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:50.038955Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-08T21:24:50.038966Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-08T21:24:49.555Z] "GET /v1/subscriptions HTTP/2" 200 - via_upstream - "-" 0 413 15 - "100.64.0.3" "curl/7.76.1" "8aa040ce-c26b-4005-9f3e-3f7fc27debbe" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.134.0.56:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.49:49066 10.134.0.49:443 100.64.0.3:17032 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T21:24:50.030Z] "POST /v1/api-keys HTTP/2" 201 - via_upstream - "-" 41 325 8 - "100.64.0.2" "curl/7.76.1" "e98e4d40-e590-4af7-80bf-2aa904e75666" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.134.0.56:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.49:49066 10.134.0.49:443 100.64.0.2:56410 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 2026-06-08T21:24:50.814332Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-08T21:24:50.851083Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-08T21:24:50.885766Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-08T21:24:50.938064Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:50.938105Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:50.949203Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:50.949350Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:50.949591Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:50.953365Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:50.953435Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-08T21:24:50.953447Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:50.979344Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:50.979388Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:50.990978Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:50.991137Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:50.991388Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:50.994682Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:50.994752Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-08T21:24:50.994763Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:51.047107Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-08T21:24:51.081890Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-08T21:24:51.142324Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-08T21:24:51.166069Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:51.166113Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:51.176946Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:51.177124Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:51.181178Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:51.181247Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-08T21:24:51.181258Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:51.214570Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-08T21:24:51.366126Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:51.366170Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:51.377645Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:51.377847Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:51.378331Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:51.378647Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:51.418142Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:51.418189Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:51.429167Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:51.429316Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:51.429771Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:51.430109Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T21:24:51.479591Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-08T21:24:50.794Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 30 314 19 - "100.64.0.4" "python-requests/2.32.5" "b6f3f44f-6b28-4413-aa39-c9e68685b224" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.134.0.56:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.49:60472 10.134.0.49:443 100.64.0.4:17068 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T21:24:50.834Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 27 311 16 - "100.64.0.3" "python-requests/2.32.5" "793c2b2f-1b03-4bce-ad8b-257b18e7f378" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.134.0.56:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.49:60472 10.134.0.49:443 100.64.0.3:17040 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T21:24:50.870Z] "GET /v1/api-keys/6513dc95-faff-4729-9dd4-4d6febd6d088 HTTP/1.1" 200 - via_upstream - "-" 0 287 14 - "100.64.0.2" "python-requests/2.32.5" "a53639ae-0713-49ef-996c-40748e8eb9b2" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.134.0.56:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.49:60472 10.134.0.49:443 100.64.0.2:56422 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T21:24:50.936Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 27 311 17 - "100.64.0.2" "python-requests/2.32.5" "dad265ea-1b32-44b2-8a0a-4f7d28171fee" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.134.0.56:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.49:49066 10.134.0.49:443 100.64.0.2:49662 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T21:24:50.977Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 27 312 17 - "100.64.0.4" "python-requests/2.32.5" "fe22fd82-1a68-4989-90a3-2e305030b230" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.134.0.56:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.49:49066 10.134.0.49:443 100.64.0.4:17078 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T21:24:51.025Z] "POST /v1/api-keys/search HTTP/1.1" 200 - via_upstream - "-" 124 1198 21 - "100.64.0.3" "python-requests/2.32.5" "49d13c01-c772-437d-972d-010215859c4f" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.134.0.56:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.49:60472 10.134.0.49:443 100.64.0.3:17054 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T21:24:51.066Z] "POST /v1/api-keys/search HTTP/1.1" 200 - via_upstream - "-" 123 330 15 - "100.64.0.2" "python-requests/2.32.5" "85a1b2e1-e2f6-4356-b2e0-6411d061d122" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.134.0.56:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.49:60472 10.134.0.49:443 100.64.0.2:56430 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T21:24:51.125Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 27 312 17 - "100.64.0.4" "python-requests/2.32.5" "d1522111-60a2-4624-99cb-2bac549379ed" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.134.0.56:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.49:60472 10.134.0.49:443 100.64.0.4:17094 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T21:24:51.164Z] "DELETE /v1/api-keys/70c95d7e-9729-4a00-943d-a81468af222a HTTP/1.1" 200 - via_upstream - "-" 0 288 16 - "100.64.0.3" "python-requests/2.32.5" "7e4fd8ec-8198-4269-9279-a449af6eb0c3" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.134.0.56:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.49:49066 10.134.0.49:443 100.64.0.3:17058 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T21:24:51.199Z] "GET /v1/api-keys/70c95d7e-9729-4a00-943d-a81468af222a HTTP/1.1" 200 - via_upstream - "-" 0 288 14 - "100.64.0.3" "python-requests/2.32.5" "a1a35cde-6526-4234-8c9f-8855a494be79" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "10.134.0.56:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.49:60472 10.134.0.49:443 100.64.0.3:49602 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T21:24:51.234Z] "POST /maas-api/v1/api-keys HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 28 0 13 - "100.64.0.2" "python-requests/2.32.5" "940c2d60-6668-497f-b593-e1382aa950b6" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "-" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local - 10.134.0.49:443 100.64.0.2:56442 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T21:24:51.315Z] "POST /maas-api/v1/api-keys HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 26 0 13 - "100.64.0.4" "python-requests/2.32.5" "c1c5096a-1c80-4a0a-8157-ae27faf42377" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "-" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local - 10.134.0.49:443 100.64.0.4:17110 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T21:24:51.364Z] "POST /maas-api/v1/api-keys HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 23 0 14 - "100.64.0.4" "python-requests/2.32.5" "accd6cef-04ab-4fae-b152-64b306211075" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "-" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local - 10.134.0.49:443 100.64.0.4:11442 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T21:24:51.416Z] "POST /maas-api/v1/api-keys/bulk-revoke HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 29 0 13 - "100.64.0.2" "python-requests/2.32.5" "5db13fc3-1e02-46c9-b0da-cf9d745cf09a" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "-" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local - 10.134.0.49:443 100.64.0.2:49664 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T21:24:51.465Z] "POST /maas-api/v1/api-keys HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 34 0 13 - "100.64.0.3" "python-requests/2.32.5" "28a5aaa8-6192-4e6f-b5b3-ba1d10166503" "maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com" "-" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local - 10.134.0.49:443 100.64.0.3:17070 maas.apps.eeb8827a-1dd0-4b97-976f-c65409b72f82.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1