2026-11-02T01:42:28.869026Z info FLAG: --concurrency="0" 2026-11-02T01:42:28.869045Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-02T01:42:28.869049Z info FLAG: --help="false" 2026-11-02T01:42:28.869051Z info FLAG: --log_as_json="false" 2026-11-02T01:42:28.869053Z info FLAG: --log_caller="" 2026-11-02T01:42:28.869055Z info FLAG: --log_output_level="default:info" 2026-11-02T01:42:28.869057Z info FLAG: --log_stacktrace_level="default:none" 2026-11-02T01:42:28.869062Z info FLAG: --log_target="[stdout]" 2026-11-02T01:42:28.869064Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-02T01:42:28.869066Z info FLAG: --outlierLogPath="" 2026-11-02T01:42:28.869068Z info FLAG: --profiling="true" 2026-11-02T01:42:28.869070Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-02T01:42:28.869072Z info FLAG: --proxyLogLevel="warning" 2026-11-02T01:42:28.869074Z info FLAG: --serviceCluster="istio-proxy" 2026-11-02T01:42:28.869076Z info FLAG: --stsPort="0" 2026-11-02T01:42:28.869078Z info FLAG: --templateFile="" 2026-11-02T01:42:28.869079Z info FLAG: --tokenManagerPlugin="" 2026-11-02T01:42:28.869082Z info FLAG: --vklog="0" 2026-11-02T01:42:28.869084Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-02T01:42:28.869250Z info Proxy role ips=[10.133.0.28] type=router id=maas-default-gateway-openshift-default-778b7b5c47-7wmnn.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-02T01:42:28.869301Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-02T01:42:28.870480Z info cpu limit detected as 2, setting concurrency 2026-11-02T01:42:28.870734Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-02T01:42:28.870745Z info JWT policy is third-party-jwt 2026-11-02T01:42:28.870747Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-02T01:42:28.872303Z info Starting default Istio SDS Server 2026-11-02T01:42:28.872324Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-02T01:42:28.872349Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-02T01:42:28.872736Z info Opening status port 15020 2026-11-02T01:42:28.873977Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-02T01:42:28.875823Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-02T01:42:28.876979Z info Starting proxy agent 2026-11-02T01:42:28.877020Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-02T01:42:28.885056Z info sds Starting SDS grpc server 2026-11-02T01:42:28.885077Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-02T01:42:28.940673Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-02T01:42:29.077862Z info cache generated new workload certificate resourceName=default latency=204.193875ms ttl=23h59m59.922142424s 2026-11-02T01:42:29.077899Z info cache Root cert has changed, start rotating root cert 2026-11-02T01:42:29.077930Z info cache returned workload trust anchor from cache ttl=23h59m59.92207006s 2026-11-02T01:42:29.090322Z info ads ADS: new connection for node:1 2026-11-02T01:42:29.090377Z info cache returned workload certificate from cache ttl=23h59m59.909624581s 2026-11-02T01:42:29.091004Z info ads ADS: new connection for node:2 2026-11-02T01:42:29.091080Z info cache returned workload trust anchor from cache ttl=23h59m59.908921143s 2026-11-02T01:42:29.168517Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T01:42:36.925529Z info Readiness succeeded in 8.059352974s 2026-11-02T01:42:36.925884Z info Envoy proxy is ready 2026-11-02T01:42:52.107937Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T01:42:55.208717Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T01:43:27.357Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 62 - "34.228.250.194,10.132.0.22" "curl/7.76.1" "48459c4e-85a0-4eff-be5d-f21a0fc5fcdd" "maas.apps.eba74347ce3ff1e2289c.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.28:443 10.132.0.22:55504 - v1-model-routing 2026-11-02T01:48:25.960737Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T01:48:25.960791Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T01:48:25.960814Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T01:48:25.960824Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-02T01:48:25.831Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 129 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "7e4de589-a30e-4c38-8fd9-413f13f46a41" "maas.apps.eba74347ce3ff1e2289c.prod.konflux-ocp-ci.dev" "10.133.0.25:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.28:43462 10.133.0.28:443 10.132.0.22:40164 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T01:48:27.700651Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T01:48:27.700690Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T01:48:27.700709Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T01:48:27.700716Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-02T01:48:27.582Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 118 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "feb8288f-bc45-4bf5-b878-8cf2935ca927" "maas.apps.eba74347ce3ff1e2289c.prod.konflux-ocp-ci.dev" "10.133.0.25:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.28:43462 10.133.0.28:443 10.132.0.22:40170 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T01:48:29.739467Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T01:48:29.739503Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T01:48:29.739517Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-02T01:48:29.739522Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 [2026-11-02T01:48:29.624Z] "DELETE /v1/api-keys/7c546f1b-2838-443b-bbd8-d946e0da787b HTTP/1.1" 200 - via_upstream - "-" 0 321 115 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "1d715bf7-d140-4dae-9084-5bcd2591676a" "maas.apps.eba74347ce3ff1e2289c.prod.konflux-ocp-ci.dev" "10.133.0.25:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.28:43462 10.133.0.28:443 10.132.0.22:40180 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T01:48:34.022392Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T01:48:34.022427Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T01:48:34.022441Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T01:48:34.022446Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-02T01:48:33.218Z] "DELETE /v1/api-keys/d4aa3ed8-7f3e-4599-85a4-3330dca2edc9 HTTP/1.1" 200 - via_upstream - "-" 0 321 803 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "6fe6d767-a99e-4054-9949-918a2c6b248d" "maas.apps.eba74347ce3ff1e2289c.prod.konflux-ocp-ci.dev" "10.133.0.25:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.28:43462 10.133.0.28:443 10.132.0.22:51976 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T01:48:36.323820Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T01:48:36.323857Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T01:48:36.323871Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T01:48:36.323876Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-02T01:48:35.294Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 1029 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "e3b54461-c524-4dec-8edb-a02c33a16321" "maas.apps.eba74347ce3ff1e2289c.prod.konflux-ocp-ci.dev" "10.133.0.25:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.28:40666 10.133.0.28:443 10.132.0.22:51978 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T01:48:45.052Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 97 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "bab271ed-67c2-4430-86f2-48fde9c1b98a" "maas.apps.eba74347ce3ff1e2289c.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.28:54368 10.133.0.28:443 10.132.0.22:42382 - v1-completions-path 2026-11-02T01:48:51.514503Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T01:48:51.514539Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T01:48:51.514552Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T01:48:51.514558Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-02T01:48:51.405Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 109 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "602000e8-a87b-4f91-9485-401b71f8588d" "maas.apps.eba74347ce3ff1e2289c.prod.konflux-ocp-ci.dev" "10.133.0.25:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.28:43462 10.133.0.28:443 10.132.0.22:42406 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T01:48:52.322248Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=26 [2026-11-02T01:48:52.249Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 288 77 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "dd88d0ed-2b9a-4eca-9205-e0838d73255a" "maas.apps.eba74347ce3ff1e2289c.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.28:53948 10.133.0.28:443 10.132.0.22:42416 - v1-completions-path 2026-11-02T01:48:55.167154Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T01:49:16.499602Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T01:54:37.349690Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T01:54:37.349728Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T01:54:37.349742Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T01:54:37.349747Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-02T01:54:37.200Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 322 148 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "5d11031e-0102-42a9-b8c3-dd819de888bb" "maas.apps.eba74347ce3ff1e2289c.prod.konflux-ocp-ci.dev" "10.133.0.25:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.28:36870 10.133.0.28:443 10.132.0.22:37116 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T01:54:37.549Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 136 338 101 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "56b36a68-5523-4ae6-943b-70967ec2b635" "maas.apps.eba74347ce3ff1e2289c.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.28:53948 10.133.0.28:443 10.132.0.22:37130 - v1-chat-completions-path 2026-11-02T01:55:05.348167Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T01:55:05.348200Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T01:55:05.348212Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T01:55:05.348217Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-02T01:55:05.336Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 91 11 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "4059d3f3-aeac-4efa-9ef7-4fa42e2d7287" "maas.apps.eba74347ce3ff1e2289c.prod.konflux-ocp-ci.dev" "10.133.0.25:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.28:36870 10.133.0.28:443 10.132.0.22:34978 - odh-ai-gateway-infra.maas-api-route.4 2026-11-02T01:55:06.219425Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T01:55:06.219464Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T01:55:06.219484Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T01:55:06.219489Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T01:55:06.105Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 113 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "c9a0cb29-25b4-4cd2-93d5-d3f94542e7b5" "maas.apps.eba74347ce3ff1e2289c.prod.konflux-ocp-ci.dev" "10.133.0.25:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.28:36870 10.133.0.28:443 10.132.0.22:34986 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T01:55:06.821845Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T01:55:06.821879Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T01:55:06.821892Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-02T01:55:06.821897Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T01:55:06.822061Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 2026-11-02T01:55:06.822106Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=26 2026-11-02T01:55:06.822112Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=26 2026-11-02T01:55:06.822143Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=26 2026-11-02T01:55:06.822147Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=26 [2026-11-02T01:55:06.745Z] "POST /v1/completions HTTP/1.1" 404 - via_upstream - "-" 94 126 77 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "1c5bbdf5-ea55-40ba-bbea-bcc9f3a05f6c" "maas.apps.eba74347ce3ff1e2289c.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.28:53948 10.133.0.28:443 10.132.0.22:34994 - v1-completions-path 2026-11-02T01:55:31.637423Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T01:56:01.405620Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T01:57:15.690795Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T01:58:11.715017Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T01:59:23.032144Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:09:36.985011Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:11:16.305340Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:11:54.427587Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:12:51.808833Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:15:24.328282Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-02T02:18:00.711917Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:18:26.721350Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:19:23.019025Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:19:44.425625Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:19:49.203260Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:20:01.317577Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-11-02T02:20:01.311Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 83 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "8ea1979c-e68c-4199-9c94-c5e6f91dd6f8" "maas.apps.eba74347ce3ff1e2289c.prod.konflux-ocp-ci.dev" "10.134.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.28:33850 10.133.0.28:443 10.132.0.22:45792 - v1-completions-path 2026-11-02T02:20:02.255624Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T02:20:02.255662Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T02:20:02.255676Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-02T02:20:02.255681Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 [2026-11-02T02:20:02.198Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 170 57 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "27b02e8c-f5a0-4f00-b397-0687fe560ed1" "maas.apps.eba74347ce3ff1e2289c.prod.konflux-ocp-ci.dev" "10.134.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.28:33854 10.133.0.28:443 10.132.0.22:45822 - v1-catch-all-path [2026-11-02T02:20:14.390Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 281 65 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "64f12391-4e33-4e63-ad71-10dfc36fee27" "maas.apps.eba74347ce3ff1e2289c.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.28:53948 10.133.0.28:443 10.132.0.22:41336 - v1-completions-path 2026-11-02T02:20:15.879700Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:20:32.806781Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T02:20:42.384Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 61 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "13f0a015-a085-4d8e-844f-28bedd1e68d6" "maas.apps.eba74347ce3ff1e2289c.prod.konflux-ocp-ci.dev" "10.134.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.28:33854 10.133.0.28:443 10.132.0.22:50388 - v1-completions-path 2026-11-02T02:21:03.925436Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:21:14.589585Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:21:21.172219Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:21:34.760953Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T02:21:34.760989Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T02:21:34.761004Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T02:21:34.761009Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T02:21:34.643Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 317 117 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "aaaec80b-0ba3-44ac-92ce-043b82d5243f" "maas.apps.eba74347ce3ff1e2289c.prod.konflux-ocp-ci.dev" "10.133.0.25:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.28:33256 10.133.0.28:443 10.132.0.22:32912 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T02:21:52.661321Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:21:57.609619Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:22:23.781659Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:22:30.411146Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:22:56.338831Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:23:18.356912Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:23:31.476558Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:23:37.895807Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:24:05.237470Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:24:10.888615Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:24:32.189108Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:24:53.651588Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:25:04.374090Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:25:23.318134Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:25:29.299312Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:29:27.719553Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:29:45.173717Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:29:49.760706Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:30:14.202205Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:32:35.428417Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:33:01.968782Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:33:27.510634Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:33:43.133766Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:33:49.984539Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:34:03.334393Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:34:08.110561Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:34:13.800176Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T02:34:13.800212Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T02:34:13.800227Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T02:34:13.800233Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T02:34:13.668Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 336 131 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "938d9ff7-ed82-435e-ae5a-d84fcce7a6c3" "maas.apps.eba74347ce3ff1e2289c.prod.konflux-ocp-ci.dev" "10.133.0.25:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.28:53500 10.133.0.28:443 10.132.0.22:47438 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T02:34:29.046513Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:37:54.826670Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:38:08.049951Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:38:14.741332Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=26 [2026-11-02T02:38:14.662Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 78 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "100b2f6e-3129-405c-8377-4cc850e6b226" "maas.apps.eba74347ce3ff1e2289c.prod.konflux-ocp-ci.dev" "10.132.0.59:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.28:50860 10.133.0.28:443 10.132.0.22:38268 - v1-catch-all-path [2026-11-02T02:38:15.425Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 429 - - "-" 61 18 55 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "465805fc-e209-4290-923d-07e951bdfcf5" "maas.apps.eba74347ce3ff1e2289c.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.28:443 10.132.0.22:38276 - v1-catch-all-path 2026-11-02T02:38:29.064030Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:38:33.778672Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:38:36.196769Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=26 [2026-11-02T02:38:34.511Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 1684 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "b5f32480-8133-4121-935c-85d72b34f8ba" "maas.apps.eba74347ce3ff1e2289c.prod.konflux-ocp-ci.dev" "10.132.0.59:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.28:50860 10.133.0.28:443 10.132.0.22:54238 - v1-catch-all-path 2026-11-02T02:38:49.961883Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:39:01.584558Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8