2026-09-15T09:16:14.776427Z info FLAG: --concurrency="0" 2026-09-15T09:16:14.776454Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-09-15T09:16:14.776458Z info FLAG: --help="false" 2026-09-15T09:16:14.776461Z info FLAG: --log_as_json="false" 2026-09-15T09:16:14.776463Z info FLAG: --log_caller="" 2026-09-15T09:16:14.776465Z info FLAG: --log_output_level="default:info" 2026-09-15T09:16:14.776467Z info FLAG: --log_stacktrace_level="default:none" 2026-09-15T09:16:14.776473Z info FLAG: --log_target="[stdout]" 2026-09-15T09:16:14.776475Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-09-15T09:16:14.776477Z info FLAG: --outlierLogPath="" 2026-09-15T09:16:14.776479Z info FLAG: --profiling="true" 2026-09-15T09:16:14.776482Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-09-15T09:16:14.776484Z info FLAG: --proxyLogLevel="warning" 2026-09-15T09:16:14.776486Z info FLAG: --serviceCluster="istio-proxy" 2026-09-15T09:16:14.776488Z info FLAG: --stsPort="0" 2026-09-15T09:16:14.776490Z info FLAG: --templateFile="" 2026-09-15T09:16:14.776492Z info FLAG: --tokenManagerPlugin="" 2026-09-15T09:16:14.776495Z info FLAG: --vklog="0" 2026-09-15T09:16:14.776498Z info Version 1.27.8-05659db4d2c3082fe25be2712780475d56cc782c-Clean 2026-09-15T09:16:14.776668Z info Proxy role ips=[10.132.0.27] type=router id=maas-default-gateway-openshift-default-5ccb968c66-cxzvg.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-09-15T09:16:14.776721Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-09-15T09:16:14.777906Z info cpu limit detected as 2, setting concurrency 2026-09-15T09:16:14.778128Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-09-15T09:16:14.778140Z info JWT policy is third-party-jwt 2026-09-15T09:16:14.778143Z info using credential fetcher of JWT type in cluster.local trust domain 2026-09-15T09:16:14.780226Z info Starting default Istio SDS Server 2026-09-15T09:16:14.780625Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-09-15T09:16:14.780649Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-09-15T09:16:14.780589Z info Opening status port 15020 2026-09-15T09:16:14.781438Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-09-15T09:16:14.781726Z info sds Starting SDS grpc server 2026-09-15T09:16:14.781749Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-09-15T09:16:14.782979Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-09-15T09:16:14.783864Z info Starting proxy agent 2026-09-15T09:16:14.783895Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-09-15T09:16:14.840569Z warning envoy main external/envoy/source/server/server.cc:901 Usage of the deprecated runtime key overload.global_downstream_max_connections, consider switching to `envoy.resource_monitors.global_downstream_max_connections` instead.This runtime key will be removed in future. thread=8 2026-09-15T09:16:14.840965Z warning envoy main external/envoy/source/server/server.cc:997 There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor. thread=8 2026-09-15T09:16:14.847114Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-09-15T09:16:14.932420Z info cache generated new workload certificate resourceName=default latency=150.539582ms ttl=23h59m59.067583896s 2026-09-15T09:16:14.932463Z info cache Root cert has changed, start rotating root cert 2026-09-15T09:16:14.932497Z info cache returned workload trust anchor from cache ttl=23h59m59.067503435s 2026-09-15T09:16:14.934434Z info ads ADS: new connection for node:1 2026-09-15T09:16:14.934481Z info cache returned workload certificate from cache ttl=23h59m59.065520468s 2026-09-15T09:16:14.934850Z info ads ADS: new connection for node:2 2026-09-15T09:16:14.934926Z info cache returned workload trust anchor from cache ttl=23h59m59.065075297s 2026-09-15T09:16:14.979094Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:16:21.193747Z info Readiness succeeded in 6.419681178s 2026-09-15T09:16:21.194044Z info Envoy proxy is ready 2026-09-15T09:16:35.064092Z info ads ADS: new connection for node:3 2026-09-15T09:16:35.064269Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-09-15T09:16:35.064308Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-09-15T09:16:35.300794Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:16:52.016404Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:16:59.628155Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:17:21.138909Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:17:28.449703Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:17:42.963811Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:18:10.172113Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:19:26.104304Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-09-15T09:19:26.075Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 320 28 - "34.228.250.194,10.132.0.11" "curl/7.76.1" "703dc884-3693-4902-9a0e-0cdcb66c4a71" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:53374 10.132.0.27:443 10.132.0.11:37780 - odh-ai-gateway-infra.maas-api-route.3 2026-09-15T09:19:26.192898Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-09-15T09:19:26.156Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 588 36 - "34.228.250.194,10.132.0.11" "curl/7.76.1" "bbef681b-254f-4118-a1b7-c54948515059" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:53374 10.132.0.27:443 10.132.0.11:37784 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:19:26.268Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 528 59 - "34.228.250.194,10.132.0.11" "curl/7.76.1" "a6090323-ddef-4493-ab9d-91c6c78d3d84" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:49592 10.132.0.27:443 10.132.0.11:37790 - llm.facebook-opt-125m-simulated-kserve-route.3 [2026-09-15T09:19:26.373Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 426 23 - "34.228.250.194,10.132.0.11" "curl/7.76.1" "6c8679d6-2b84-4164-b2cc-2570b2fde425" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:49604 10.132.0.27:443 10.132.0.11:44550 - llm.facebook-opt-125m-simulated-kserve-route.3 [2026-09-15T09:19:26.422Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 388 17 - "34.228.250.194,10.132.0.11" "curl/7.76.1" "83b85807-6e5a-4dec-b11e-ec4e4a5b263d" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:49604 10.132.0.27:443 10.132.0.11:44556 - llm.facebook-opt-125m-simulated-kserve-route.3 [2026-09-15T09:19:26.474Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 14 - "34.228.250.194,10.132.0.11" "curl/7.76.1" "ba5c2507-30a3-448a-b678-2dd4e7d68a68" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:44564 - llm.facebook-opt-125m-simulated-kserve-route.3 [2026-09-15T09:19:26.523Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 14 - "34.228.250.194,10.132.0.11" "curl/7.76.1" "54e3e31d-5828-4e6a-b3f7-581f56155607" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:44566 - llm.facebook-opt-125m-simulated-kserve-route.3 [2026-09-15T09:19:26.569Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 14 - "34.228.250.194,10.132.0.11" "curl/7.76.1" "133b3e23-5633-4edc-b790-fa92c180462e" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:44572 - llm.facebook-opt-125m-simulated-kserve-route.3 [2026-09-15T09:19:26.607Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 14 - "34.228.250.194,10.132.0.11" "curl/7.76.1" "b1c090d0-db08-41be-9424-0f40e703fc6a" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:44586 - llm.facebook-opt-125m-simulated-kserve-route.3 [2026-09-15T09:19:26.653Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 14 - "34.228.250.194,10.132.0.11" "curl/7.76.1" "2af3f48a-4916-4789-8331-7788c806e1ea" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:44588 - llm.facebook-opt-125m-simulated-kserve-route.3 [2026-09-15T09:19:26.692Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 14 - "34.228.250.194,10.132.0.11" "curl/7.76.1" "87990e2b-9334-487d-9864-baf51830e79e" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:44592 - llm.facebook-opt-125m-simulated-kserve-route.3 [2026-09-15T09:19:26.741Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 14 - "34.228.250.194,10.132.0.11" "curl/7.76.1" "9df2e41b-9aca-4588-be8b-e56a7660d0c0" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:44602 - llm.facebook-opt-125m-simulated-kserve-route.3 2026-09-15T09:19:26.866541Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-09-15T09:19:26.871323Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-09-15T09:19:27.142762Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-09-15T09:19:27.187310Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-09-15T09:19:26.788Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 14 - "34.228.250.194,10.132.0.11" "curl/7.76.1" "15829b02-9eb2-47f8-a83e-fdb34efce52c" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:44610 - llm.facebook-opt-125m-simulated-kserve-route.3 [2026-09-15T09:19:26.825Z] "POST /v1/chat/completions HTTP/1.1" 401 - - "-" 122 0 5 - "34.228.250.194,10.132.0.11" "curl/7.76.1" "c241cd09-5310-458a-a32e-3edffd0d3023" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:44622 - llm.facebook-opt-125m-simulated-kserve-route.3 [2026-09-15T09:19:26.854Z] "DELETE /v1/api-keys/6a3a1ca1-a3f5-4690-a81d-12b7d72da6e1 HTTP/1.1" 200 - via_upstream - "-" 0 365 16 - "34.228.250.194,10.132.0.11" "curl/7.76.1" "9f4a5b24-d4f5-4126-a30b-e63877c5194d" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:58324 10.132.0.27:443 10.132.0.11:44634 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:19:27.138Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 20 4 - "34.228.250.194,10.132.0.11" "curl/7.76.1" "e8696d78-e62f-4d93-bca2-d1e802d9f840" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:53374 10.132.0.27:443 10.132.0.11:44640 - odh-ai-gateway-infra.maas-api-route.4 [2026-09-15T09:19:27.174Z] "GET /v1/subscriptions HTTP/1.1" 200 - via_upstream - "-" 0 488 13 - "34.228.250.194,10.132.0.11" "curl/7.76.1" "231dc348-55b0-4167-846f-20d4f4fa4c0e" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:53374 10.132.0.27:443 10.132.0.11:44652 - odh-ai-gateway-infra.maas-api-route.3 2026-09-15T09:19:41.709465Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-09-15T09:19:41.782350Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-09-15T09:19:41.704Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 20 5 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "3b347611-64a0-4266-93b6-c7edcd27762d" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:53374 10.132.0.27:443 10.132.0.11:38932 - odh-ai-gateway-infra.maas-api-route.4 [2026-09-15T09:19:41.736Z] "POST /maas-api/v1/tokens HTTP/1.1" 401 - - "-" 20 0 6 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "05146c83-a284-484f-b523-6a8c5aec9cf4" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.132.0.27:443 10.132.0.11:38936 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:19:41.769Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 588 12 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "5a2b367b-cfe7-4846-8dbf-3dd938b402cc" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:53374 10.132.0.27:443 10.132.0.11:38942 - odh-ai-gateway-infra.maas-api-route.3 2026-09-15T09:19:41.825034Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-09-15T09:19:41.868398Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-09-15T09:19:41.868555Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 2026-09-15T09:19:41.885448Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-09-15T09:19:41.910914Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-09-15T09:19:41.911084Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 2026-09-15T09:19:41.947131Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-09-15T09:19:42.148646Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-09-15T09:19:42.191877Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-09-15T09:19:42.289292Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-09-15T09:19:42.323220Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-09-15T09:19:42.326214Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-09-15T09:19:42.367979Z info ads ADS: new connection for node:4 2026-09-15T09:19:42.370746Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-09-15T09:19:42.370794Z info cache read certificate from file resource=file-root:system 2026-09-15T09:19:42.537512Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-09-15T09:19:42.577253Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-09-15T09:19:42.642300Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-09-15T09:19:42.677524Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-09-15T09:19:41.810Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 321 14 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "83a55168-a75e-4e57-a522-7e75505aa23d" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:58324 10.132.0.27:443 10.132.0.11:38950 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:19:41.850Z] "POST /v1/chat/completions HTTP/1.1" 404 - via_upstream - "-" 111 126 18 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "e11ef722-0699-4a54-ab64-2901321b2904" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:49604 10.132.0.27:443 10.132.0.11:38962 - llm.facebook-opt-125m-simulated-kserve-route.2 [2026-09-15T09:19:41.872Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 84 327 12 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "009f55e9-f988-4a1d-9274-1b68aae2ca7d" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:53374 10.132.0.27:443 10.132.0.11:38972 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:19:41.890Z] "POST /v1/completions HTTP/1.1" 404 - via_upstream - "-" 94 126 20 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "88aa6b67-4f00-4b86-b9d5-d68c386a3391" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:49604 10.132.0.27:443 10.132.0.11:38982 - llm.facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:19:41.906Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 83 0 7 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "44f5d1f3-0c97-4590-b0dd-7ff9bf0b8ab2" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.132.0.27:443 10.132.0.11:38994 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:19:41.933Z] "DELETE /v1/api-keys/b4a8927e-da94-41a5-b6e8-832f824ef148 HTTP/1.1" 200 - via_upstream - "-" 0 370 13 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "8db9d07c-ce3c-424e-a4f5-2e83abfe5863" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:58324 10.132.0.27:443 10.132.0.11:39008 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:19:42.135Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 13 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "9f2a17f5-bbf4-4010-87c5-52ea8658e6b3" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:53374 10.132.0.27:443 10.132.0.11:39018 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:19:42.178Z] "DELETE /v1/api-keys/d71525da-9bbc-4616-a5a8-2bb99ffa2144 HTTP/1.1" 200 - via_upstream - "-" 0 321 13 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "bea03cb5-9081-47e5-b04a-d82d8307be57" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:58324 10.132.0.27:443 10.132.0.11:39026 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:19:42.223Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 9 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "4cd9553c-0d10-4c91-910f-7e5f01418163" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.132.0.27:443 10.132.0.11:39042 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:19:42.275Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 13 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "88370d65-7ccd-4308-9b02-8ab35071c7de" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:53374 10.132.0.27:443 10.132.0.11:39044 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:19:42.312Z] "DELETE /v1/api-keys/aaf3941c-dce1-4a21-8543-245d913c29f5 HTTP/1.1" 200 - via_upstream - "-" 0 321 14 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "d4a9bef8-a6fc-4b67-880b-70c8f5b8cab4" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:53374 10.132.0.27:443 10.132.0.11:39050 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:19:42.523Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 14 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "b3be127f-13bf-4d6d-a5e3-ff71bda82531" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:58324 10.132.0.27:443 10.132.0.11:39058 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:19:42.560Z] "DELETE /v1/api-keys/e4475f89-1d83-46bd-9d1d-02f540ad0ade HTTP/1.1" 200 - via_upstream - "-" 0 321 16 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "f6d7b636-e3bf-49ec-b415-c708c362a24e" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:58324 10.132.0.27:443 10.132.0.11:39070 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:19:42.598Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 10 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "ae0da120-85b2-41a2-9cbe-39a5262fa0ae" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.132.0.27:443 10.132.0.11:39078 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:19:42.628Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 13 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "c86cc271-68e0-4ce7-af0b-49af1c417a01" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:53374 10.132.0.27:443 10.132.0.11:39084 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:19:42.661Z] "DELETE /v1/api-keys/6d5adda4-498e-40f5-9bbb-d05c564e4458 HTTP/1.1" 200 - via_upstream - "-" 0 321 15 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "bd2eb55b-b265-4386-9900-22ca56ac9622" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:53374 10.132.0.27:443 10.132.0.11:39094 - odh-ai-gateway-infra.maas-api-route.3 2026-09-15T09:19:42.868173Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-09-15T09:19:42.905046Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-09-15T09:19:42.973729Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-09-15T09:19:43.003753Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-09-15T09:19:43.192678Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-09-15T09:19:43.407885Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-09-15T09:19:43.530961Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-09-15T09:19:42.854Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 13 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "8ccb4ed5-8e4e-4b7c-9191-d3a48a2d40ce" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:58324 10.132.0.27:443 10.132.0.11:39106 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:19:42.890Z] "DELETE /v1/api-keys/3047cc58-4a91-43d0-b1e9-0bab130dab56 HTTP/1.1" 200 - via_upstream - "-" 0 321 14 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "ee378b23-86e7-4082-8f40-af5fd05f299a" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:53374 10.132.0.27:443 10.132.0.11:39114 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:19:42.928Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 11 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "84998364-f35c-4dfe-b31e-cf5c9862bac8" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.132.0.27:443 10.132.0.11:39120 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:19:42.960Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 13 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "a430eca2-082a-4b09-98df-5d9fc2edb485" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:58324 10.132.0.27:443 10.132.0.11:39134 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:19:42.991Z] "DELETE /v1/api-keys/fdeea101-293b-4c3e-a016-210868c543fc HTTP/1.1" 200 - via_upstream - "-" 0 321 12 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "6340dbcf-a8f4-42eb-93b4-cacbc0702886" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:58324 10.132.0.27:443 10.132.0.11:39146 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:19:43.178Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 13 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "31e1eec6-98f9-4e75-ba63-a89d09887c27" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:58324 10.132.0.27:443 10.132.0.11:39160 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:19:43.215Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 13 11 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "bc8ea5aa-27b5-467f-8217-1dfdf7ab494c" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:39166 - llm.facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:19:43.393Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 14 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "f68bea62-6e20-4524-8c4a-a33285eff858" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:58324 10.132.0.27:443 10.132.0.11:39180 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:19:43.428Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 19 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "ebc3187c-cb20-4ecf-a81b-4e158e2d3873" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:49592 10.132.0.27:443 10.132.0.11:39182 - llm.facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:19:43.470Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 20 - "34.228.250.194,10.132.0.11" "-" "261dbf90-12e8-4073-bd12-6c2158e3dcf0" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:49592 10.132.0.27:443 10.132.0.11:39198 - llm.facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:19:43.516Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 316 14 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "3fc0aab9-5e05-49bc-a313-edb6e61f5b76" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:58324 10.132.0.27:443 10.132.0.11:39206 - odh-ai-gateway-infra.maas-api-route.3 2026-09-15T09:19:46.873063Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:19:47.782479Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-09-15T09:19:47.769Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 322 13 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "b9eab1c3-569e-41a3-834b-33a8d0fc9c48" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:53374 10.132.0.27:443 10.132.0.11:56462 - odh-ai-gateway-infra.maas-api-route.3 2026-09-15T09:19:48.651237Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-09-15T09:19:48.651383Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-09-15T09:19:48.177Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 403 - - "-" 83 0 10 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "3a99f2bf-05a3-44a0-84e1-fc1223a0095d" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|443||httpbingo.org - 10.132.0.27:443 10.132.0.11:56464 - llm.e2e-external-model.0 [2026-09-15T09:19:48.211Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 6 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "992fa22d-908c-4737-8e8e-1fcbba994cd3" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|443||httpbingo.org - 10.132.0.27:443 10.132.0.11:56472 - llm.e2e-external-model.0 [2026-09-15T09:19:48.554Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 11 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "86a2d888-2c12-4c99-a98d-d80fcaf2ea3b" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:56504 - llm.facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:19:48.241Z] "POST /v1/chat/completions HTTP/1.1" 404 - via_upstream - "-" 83 39 410 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "21d90d97-da13-4eef-a7c7-2d28a0672064" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.27:57250 10.132.0.27:443 10.132.0.11:56488 - llm.e2e-external-model.1 [2026-09-15T09:19:51.472Z] "GET /v1/tenants HTTP/1.1" 404 NR route_not_found - "-" 0 0 5 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "91d3bd01-0d29-4774-a108-a4f5269b122b" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" - - 10.132.0.27:443 10.132.0.11:56514 - - 2026-09-15T09:19:54.545271Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-09-15T09:20:01.400Z] "POST /llm/nonexistent-model-xyz/v1/chat/completions HTTP/1.1" 404 NR route_not_found - "-" 86 0 9 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "fb8ad045-af25-4790-97a7-b4c250f6037e" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" - - 10.132.0.27:443 10.132.0.11:47784 - - 2026-09-15T09:20:04.748436Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:20:05.789166Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-09-15T09:20:06.164718Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-09-15T09:20:06.164968Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 2026-09-15T09:20:06.263745Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-09-15T09:20:06.264007Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-09-15T09:20:05.743Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 45 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "bf0a4f43-2af4-4a42-b0a7-7eec69bfc27b" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:53374 10.132.0.27:443 10.132.0.11:47802 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:20:05.812Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 76 107 59 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "a2dcd85e-8a78-42b7-9b45-81b73e39a18c" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:47818 - llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:20:05.145Z] "POST /v1/chat/completions HTTP/1.1" 404 - via_upstream - "-" 83 39 1019 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "05888252-d47d-4e8e-b98a-0798c1ccdf53" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.27:33322 10.132.0.27:443 10.132.0.11:47792 - llm.e2e-external-model.1 [2026-09-15T09:20:06.188Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 404 - via_upstream - "-" 82 39 75 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "2e806d88-dce0-4d8b-b453-dd79d12689a6" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.27:33322 10.132.0.27:443 10.132.0.11:47826 - llm.e2e-external-model.2 2026-09-15T09:20:06.649667Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-09-15T09:20:06.649948Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-09-15T09:20:06.289Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 404 - via_upstream - "-" 52 39 360 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "ca18792a-3a8c-4ba4-8395-9b02d4f02bdd" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.27:33322 10.132.0.27:443 10.132.0.11:47840 - llm.e2e-external-model.2 2026-09-15T09:20:07.771065Z info ads ADS: "" 4 terminated with stream closed 2026-09-15T09:20:09.812540Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-09-15T09:20:09.799Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 13 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "3cdc4d7e-4e7b-464e-9442-873c843c6b6a" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:53374 10.132.0.27:443 10.132.0.11:48582 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:20:09.843Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 289 43 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "808ccca7-9567-4a09-bd9d-35b47f151029" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:49604 10.132.0.27:443 10.132.0.11:48598 - llm.facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:20:09.904Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 27 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "2fedec44-2ec3-4950-949d-c40826c25718" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:49604 10.132.0.27:443 10.132.0.11:48600 - llm.facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:20:09.954Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 299 21 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "d1d9c7e9-16d0-4ce9-9b3d-a14ad9b4464c" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:49604 10.132.0.27:443 10.132.0.11:48612 - llm.facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:20:09.998Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 31 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "24b8e16e-227a-41d7-8061-3d1582c34c83" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:49592 10.132.0.27:443 10.132.0.11:48624 - llm.facebook-opt-125m-simulated-kserve-route.0 2026-09-15T09:20:13.489451Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:20:24.358223Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-09-15T09:20:23.951Z] "POST /maas-api/internal/v1/subscriptions/select HTTP/1.1" 404 NR route_not_found - "-" 56 0 14 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "13ebf19b-8c20-4f1a-9b93-3a3e7586e749" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" - - 10.132.0.27:443 10.132.0.11:41418 - - [2026-09-15T09:20:24.128Z] "POST /maas-api/internal/v1/api-keys/cleanup HTTP/1.1" 404 NR route_not_found - "-" 2 0 14 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "07e4d608-21f5-4df6-9a95-c53834a0ee7a" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" - - 10.132.0.27:443 10.132.0.11:41420 - - [2026-09-15T09:20:24.306Z] "POST /maas-api/internal/v1/api-keys/validate HTTP/1.1" 404 NR route_not_found - "-" 31 0 5 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "f34d89cf-eabc-4e66-9ba8-0781ded5d58a" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" - - 10.132.0.27:443 10.132.0.11:41422 - - [2026-09-15T09:20:24.344Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 20 14 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "9d25b890-1756-4c3d-9f29-faf1f3d3c69d" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:58324 10.132.0.27:443 10.132.0.11:41428 - odh-ai-gateway-infra.maas-api-route.4 2026-09-15T09:20:24.512752Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-09-15T09:20:24.498Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 588 14 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "15e5836e-b007-464a-9052-f8a81c97acdf" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:58324 10.132.0.27:443 10.132.0.11:41438 - odh-ai-gateway-infra.maas-api-route.3 2026-09-15T09:30:17.854573Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:30:30.893597Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:37:40.239928Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-09-15T09:37:40.215Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 69 312 24 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "bc072001-6f90-4ff7-8410-2627ecfbfd62" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:48408 10.132.0.27:443 10.132.0.11:55078 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:37:40.262Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 110 321 31 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "e84ef6db-20cd-4490-9b92-70a2a8166043" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:49604 10.132.0.27:443 10.132.0.11:55094 - llm.facebook-opt-125m-simulated-kserve-route.2 [2026-09-15T09:37:40.316Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 110 318 18 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "b6acc823-b922-4747-ad34-27baf84671ad" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:49604 10.132.0.27:443 10.132.0.11:55098 - llm.facebook-opt-125m-simulated-kserve-route.2 2026-09-15T09:41:03.049996Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-09-15T09:41:03.033Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 30 314 16 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "99812691-af8e-495b-8328-4e289d14bef0" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:41062 10.132.0.27:443 10.132.0.11:40384 - odh-ai-gateway-infra.maas-api-route.3 2026-09-15T09:41:05.151878Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-09-15T09:41:05.679716Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-09-15T09:41:05.136Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 15 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "66b52c7e-a926-42f7-a5c6-1bbe3150f80a" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:41068 10.132.0.27:443 10.132.0.11:40386 - odh-ai-gateway-infra.maas-api-route.3 2026-09-15T09:42:26.458988Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:42:26.859233Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-09-15T09:42:26.843Z] "POST /v1/api-keys HTTP/1.1" 400 - via_upstream - "-" 80 94 15 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "d5ce72ef-594d-459e-935b-e7a7f949401f" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:35402 10.132.0.27:443 10.132.0.11:47784 - odh-ai-gateway-infra.maas-api-route.3 2026-09-15T09:42:52.580746Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:42:56.035816Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-09-15T09:42:56.009Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 87 329 26 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "c51ac6ea-086c-4952-87a8-146cfbfdfef3" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:35402 10.132.0.27:443 10.132.0.11:60706 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:42:56.059Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 43 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "e40b2332-6f82-411a-ad23-9b60653575cc" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.61:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:55650 10.132.0.27:443 10.132.0.11:60720 - llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:42:56.227Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 18 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "b38754f3-bebb-442f-8833-51c98a4ff723" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.61:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:55650 10.132.0.27:443 10.132.0.11:60732 - llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:42:56.367Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 18 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "e79e51f4-617b-4b96-82e4-805fd69605aa" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.61:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:55650 10.132.0.27:443 10.132.0.11:47946 - llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:42:56.518Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 19 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "8558f036-c465-442c-a9e3-09f83fea80af" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.61:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:59134 10.132.0.27:443 10.132.0.11:47952 - llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:42:56.659Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 18 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "e6e91063-36b5-4bc6-a554-c43c241dc0a2" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.61:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:55650 10.132.0.27:443 10.132.0.11:47956 - llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:42:56.823Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 15 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "38213810-b9a3-4099-989c-ae150269d7d0" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:47968 - llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 2026-09-15T09:43:00.864500Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:43:05.488258Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-09-15T09:43:05.473Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 335 14 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "cba67c6e-354e-4f1b-93e3-216082141acb" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:35402 10.132.0.27:443 10.132.0.11:47976 - odh-ai-gateway-infra.maas-api-route.3 2026-09-15T09:43:05.667717Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-09-15T09:43:06.361823Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-09-15T09:43:05.514Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 19 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "bbd1dda9-0caa-4bbb-a6e3-724311e628ee" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.61:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:55650 10.132.0.27:443 10.132.0.11:47980 - llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:43:05.553Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 298 17 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "ad048c25-783f-4f9a-9c45-e5e40d38cfaf" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.61:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:59134 10.132.0.27:443 10.132.0.11:47984 - llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:43:05.588Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 14 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "8f9738c9-388c-4b11-8be4-0e7c9774ceac" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:47990 - llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:43:05.620Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 14 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "29e4eb24-d5f6-4778-8003-291126ce8f8c" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:48000 - llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:43:05.651Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 170 15 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "1a6edcaf-84c2-4db3-a4cf-340d9a392a1f" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.61:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:59134 10.132.0.27:443 10.132.0.11:48002 - llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.6 [2026-09-15T09:43:06.346Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 74 316 15 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "ea8dc658-2a24-4264-81cb-b1d670f3259b" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:35402 10.132.0.27:443 10.132.0.11:45156 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:43:06.382Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 283 18 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "1f372a7d-7742-4bc8-a1e0-27246f58a070" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:49592 10.132.0.27:443 10.132.0.11:45172 - llm.facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:43:07.305Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 290 18 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "45c30b8d-06c5-42cd-b77e-07d3c71a4efe" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:49592 10.132.0.27:443 10.132.0.11:45180 - llm.facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:43:08.002Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 18 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "a96d7526-9328-4741-83f8-1a3a1ea8d962" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:49604 10.132.0.27:443 10.132.0.11:45194 - llm.facebook-opt-125m-simulated-kserve-route.0 2026-09-15T09:43:08.660500Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-09-15T09:43:09.039Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 19 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "307f0cce-2197-4cd4-9af5-b58beedb4cfb" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:49604 10.132.0.27:443 10.132.0.11:45202 - llm.facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:43:14.412Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 32 11 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "2ba50493-53ef-451f-90ae-a7015469bd2e" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:45210 - llm.facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:43:19.464Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 76 107 11 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "acdf75d9-cc1d-4796-8f7c-164acd84d76a" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:49958 - llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 2026-09-15T09:43:20.238471Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-09-15T09:43:20.223Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 14 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "120570c8-94d0-4033-a9f4-2eea271a746a" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:35402 10.132.0.27:443 10.132.0.11:49972 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:43:20.598Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 32 10 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "581ab487-2eed-44e6-9e9f-d51b244f12d1" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:49984 - llm.facebook-opt-125m-simulated-kserve-route.0 2026-09-15T09:43:23.267961Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:43:28.457537Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:43:28.525035Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-09-15T09:43:28.509Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 33 328 15 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "26db17a2-6af6-4014-96d9-6dfddeed39fd" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:35402 10.132.0.27:443 10.132.0.11:39636 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:43:28.546Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 66 18 20 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "5ebe62c5-b109-469c-9f7f-ecb0598e40e7" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:39644 - llm.facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:43:30.593Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 289 19 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "0ee3b793-42f5-4d5e-98e3-593f0dc81f8b" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:49592 10.132.0.27:443 10.132.0.11:39658 - llm.facebook-opt-125m-simulated-kserve-route.0 2026-09-15T09:43:31.976808Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-09-15T09:43:31.963Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 80 322 13 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "2d55460d-6e60-4a2b-bee0-2f458606259d" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:35402 10.132.0.27:443 10.132.0.11:39670 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:43:32.337Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 32 11 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "3bfa4ac3-18fe-4aea-bc9d-59ecb5e0204c" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:39674 - llm.facebook-opt-125m-simulated-kserve-route.0 2026-09-15T09:43:33.474113Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:44:40.420848Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-09-15T09:44:40.404Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 16 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "bba4c3eb-ba77-43b6-914b-096f0423596b" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:34812 10.132.0.27:443 10.132.0.11:46714 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:44:40.441Z] "POST /llm/e2e-trlp-test-simulated/v1/completions HTTP/1.1" 403 - - "-" 73 49 21 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "f0e3a298-5d27-479e-92d0-cbb92b01cfd4" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||e2e-trlp-test-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:46726 - llm.e2e-trlp-test-simulated-kserve-route.0 2026-09-15T09:44:54.141968Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-09-15T09:45:01.262361Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:45:02.776857Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-09-15T09:45:02.161Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 73 291 33 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "d58e671f-e33a-4379-ab7b-170c5148e27c" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.134.0.22:8000" outbound|8000||e2e-trlp-test-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:34944 10.132.0.27:443 10.132.0.11:35858 - llm.e2e-trlp-test-simulated-kserve-route.0 [2026-09-15T09:45:02.763Z] "DELETE /v1/api-keys/e2e-trlp-test-key HTTP/1.1" 404 - via_upstream - "-" 0 29 13 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "62fb3f61-8b0b-4d8d-a995-961a6528ef33" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:34812 10.132.0.27:443 10.132.0.11:35860 - odh-ai-gateway-infra.maas-api-route.3 2026-09-15T09:45:05.176584Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:45:09.356944Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:45:11.196206Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-09-15T09:45:11.344970Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-09-15T09:45:11.181Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 40 329 14 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "bde195e0-49ec-4dca-84a9-21cbb5a95e64" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:34812 10.132.0.27:443 10.132.0.11:56306 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:45:11.331Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 479 13 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "6072728d-4f42-427b-a1ec-37b3d3d336b2" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:58070 10.132.0.27:443 10.132.0.11:56308 - odh-ai-gateway-infra.maas-api-route.3 2026-09-15T09:45:15.473063Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:45:37.572066Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:45:48.074659Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:46:06.686086Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-09-15T09:46:06.721012Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-09-15T09:46:06.670Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 80 322 16 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "19082683-4708-4e06-a3b8-4ef3e83ac93b" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:58070 10.132.0.27:443 10.132.0.11:39268 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:46:06.709Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 670 11 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "cac6bfa1-7b49-4176-ae23-7937003a28d2" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:58070 10.132.0.27:443 10.132.0.11:39278 - odh-ai-gateway-infra.maas-api-route.3 2026-09-15T09:46:10.569793Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:46:13.773013Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:46:14.442674Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-09-15T09:46:14.476232Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-09-15T09:46:14.425Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 84 326 17 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "875209ba-e5ac-4aa8-a34c-9a7d6120d92f" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:53118 10.132.0.27:443 10.132.0.11:39290 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:46:14.465Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 11 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "77b785c4-7c6b-4b27-942a-739918f5b30e" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:58070 10.132.0.27:443 10.132.0.11:39300 - odh-ai-gateway-infra.maas-api-route.3 2026-09-15T09:46:17.766797Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:46:23.257406Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-09-15T09:46:23.291130Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-09-15T09:46:23.241Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 88 330 15 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "40712fe1-e579-4915-a0df-42aa3f5140e6" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:58070 10.132.0.27:443 10.132.0.11:55830 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:46:23.279Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 538 11 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "f0ae4b98-60a2-40e2-8d21-3f759242589a" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:53118 10.132.0.27:443 10.132.0.11:55832 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:46:23.310Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 21 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "79281e9d-2ec7-42c7-9ee2-24c53ba71c26" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.61:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:55650 10.132.0.27:443 10.132.0.11:55838 - llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:46:23.352Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 15 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "b7c0c0c2-3d77-4d98-abd0-8f7c4533a037" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:55846 - llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-09-15T09:46:23.386Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 22 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "71adbb27-76cc-45a2-a4d0-61e893ec5461" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:55852 - llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 2026-09-15T09:46:23.441673Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-09-15T09:46:23.430Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 538 10 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "1a61c87e-0dd4-452d-b5d5-f6f6c5f11a7d" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:58070 10.132.0.27:443 10.132.0.11:55858 - odh-ai-gateway-infra.maas-api-route.3 2026-09-15T09:46:25.879479Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-09-15T09:46:30.820Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 403 - - "-" 61 39 15 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "8c6fcfca-f393-40f0-9457-f5bc1ca73b25" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:35630 - llm.e2e-embedding-simulated-kserve-route.6 2026-09-15T09:46:33.260749Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:46:36.875371Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-09-15T09:46:36.853Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 317 21 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "7baf50b6-7751-4ec7-9c19-c3d74bec9fac" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.132.0.79:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:40740 10.132.0.27:443 10.132.0.11:51108 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:46:36.899Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 33 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "07a78bbc-2857-43ec-8a8d-c43a7ec470a1" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.60:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:46070 10.132.0.27:443 10.132.0.11:51112 - llm.e2e-embedding-simulated-kserve-route.6 [2026-09-15T09:46:37.056Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 19 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "3c8c8a67-96a1-4ecc-b4f7-3dc101b7e875" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.60:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:46070 10.132.0.27:443 10.132.0.11:51126 - llm.e2e-embedding-simulated-kserve-route.6 [2026-09-15T09:46:37.194Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 19 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "f2a76637-f864-4d0c-85dc-f84f2e04ae33" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.60:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:46070 10.132.0.27:443 10.132.0.11:51128 - llm.e2e-embedding-simulated-kserve-route.6 [2026-09-15T09:46:37.331Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 429 - - "-" 61 18 15 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "c5ea68c2-1b1f-4c0c-b9d7-8068ca9b68cc" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.27:443 10.132.0.11:51140 - llm.e2e-embedding-simulated-kserve-route.6 2026-09-15T09:46:40.058550Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:46:46.058413Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-09-15T09:46:49.937437Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-09-15T09:46:49.922Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 73 315 15 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "3ebdc2d0-351e-4703-9a29-ad585cbc1bf9" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.132.0.79:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.27:40740 10.132.0.27:443 10.132.0.11:49318 - odh-ai-gateway-infra.maas-api-route.3 [2026-09-15T09:46:49.960Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 34 - "34.228.250.194,10.132.0.11" "python-requests/2.32.5" "80216931-17aa-46f4-90f1-b9b1a2f4dede" "maas.apps.142e7835-858e-48ad-83cc-792d318363ed.prod.konfluxeaas.com" "10.133.0.60:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.27:49716 10.132.0.27:443 10.132.0.11:49330 - llm.e2e-embedding-simulated-kserve-route.6 2026-09-15T09:46:53.169088Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8