2026-11-04T01:16:11.260180Z info FLAG: --concurrency="0" 2026-11-04T01:16:11.260198Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-04T01:16:11.260202Z info FLAG: --help="false" 2026-11-04T01:16:11.260204Z info FLAG: --log_as_json="false" 2026-11-04T01:16:11.260206Z info FLAG: --log_caller="" 2026-11-04T01:16:11.260208Z info FLAG: --log_output_level="default:info" 2026-11-04T01:16:11.260209Z info FLAG: --log_stacktrace_level="default:none" 2026-11-04T01:16:11.260214Z info FLAG: --log_target="[stdout]" 2026-11-04T01:16:11.260216Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-04T01:16:11.260218Z info FLAG: --outlierLogPath="" 2026-11-04T01:16:11.260220Z info FLAG: --profiling="true" 2026-11-04T01:16:11.260222Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-04T01:16:11.260223Z info FLAG: --proxyLogLevel="warning" 2026-11-04T01:16:11.260225Z info FLAG: --serviceCluster="istio-proxy" 2026-11-04T01:16:11.260227Z info FLAG: --stsPort="0" 2026-11-04T01:16:11.260229Z info FLAG: --templateFile="" 2026-11-04T01:16:11.260231Z info FLAG: --tokenManagerPlugin="" 2026-11-04T01:16:11.260233Z info FLAG: --vklog="0" 2026-11-04T01:16:11.260236Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-04T01:16:11.260407Z info Proxy role ips=[10.133.0.46] type=router id=maas-default-gateway-openshift-default-75d7c785f-7rw6b.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-04T01:16:11.260446Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-04T01:16:11.261480Z info cpu limit detected as 2, setting concurrency 2026-11-04T01:16:11.261728Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-04T01:16:11.261740Z info JWT policy is third-party-jwt 2026-11-04T01:16:11.261743Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-04T01:16:11.263462Z info Starting default Istio SDS Server 2026-11-04T01:16:11.263481Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-04T01:16:11.263498Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-04T01:16:11.264138Z info Opening status port 15020 2026-11-04T01:16:11.264704Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-04T01:16:11.266152Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-04T01:16:11.267007Z info Starting proxy agent 2026-11-04T01:16:11.267046Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-04T01:16:11.276600Z info sds Starting SDS grpc server 2026-11-04T01:16:11.276617Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-04T01:16:11.329957Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-04T01:16:11.424439Z info ads ADS: new connection for node:1 2026-11-04T01:16:11.424867Z info ads ADS: new connection for node:2 2026-11-04T01:16:11.649710Z info cache generated new workload certificate resourceName=default latency=385.300628ms ttl=23h59m59.350296749s 2026-11-04T01:16:11.649756Z info cache Root cert has changed, start rotating root cert 2026-11-04T01:16:11.649805Z info cache returned workload trust anchor from cache ttl=23h59m59.350196248s 2026-11-04T01:16:11.649830Z info cache returned workload certificate from cache ttl=23h59m59.350171411s 2026-11-04T01:16:11.649943Z info cache returned workload trust anchor from cache ttl=23h59m59.350064553s 2026-11-04T01:16:11.650364Z info cache returned workload trust anchor from cache ttl=23h59m59.349637906s 2026-11-04T01:16:11.699406Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:16:17.898308Z info Readiness succeeded in 6.641088863s 2026-11-04T01:16:17.898611Z info Envoy proxy is ready 2026-11-04T01:16:32.593793Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:16:34.284136Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:16:35.909689Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:16:37.994038Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:16:40.307912Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:16:43.464295Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:16:46.291821Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:17:03.831955Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:17:04.322279Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:17:41.745299Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:17:41.745336Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:17:41.745349Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T01:17:41.745353Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-04T01:17:41.727Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 91 18 - "44.212.242.249,10.132.0.15" "curl/7.76.1" "9d6c7aba-7203-4b33-bbcb-899863a98498" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "10.134.0.21:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.46:58332 10.133.0.46:443 10.132.0.15:58224 - odh-ai-gateway-infra.maas-api-route.4 2026-11-04T01:19:57.753550Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:19:57.753586Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:19:57.753598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:19:57.753603Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T01:19:57.602Z] "DELETE /v1/api-keys/7dba366b-2be8-4edf-955e-5d5ea3029488 HTTP/1.1" 200 - via_upstream - "-" 0 321 150 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "52a76a3c-b434-4d72-a650-6d29ab998a0d" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "10.134.0.21:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.46:58210 10.133.0.46:443 10.132.0.15:56992 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T01:19:57.819Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 88 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "fc1f66c3-2b8b-441a-8376-264b66cfecfc" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.133.0.46:443 10.132.0.15:56994 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T01:19:58.244511Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:19:58.244558Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:19:58.244576Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:19:58.244583Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-04T01:19:58.127Z] "DELETE /v1/api-keys/f62a2f34-d6cb-4730-88a1-94948a1ce812 HTTP/1.1" 200 - via_upstream - "-" 0 321 116 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "0fdc5312-a289-4890-a877-217e1154b2a5" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "10.134.0.21:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.46:58224 10.133.0.46:443 10.132.0.15:56996 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T01:20:02.212232Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:20:02.212268Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:20:02.212281Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:20:02.212286Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-04T01:20:02.134Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 78 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "dff8f0e4-c240-4ce5-bd97-0a5c79acd665" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "10.134.0.21:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.46:58224 10.133.0.46:443 10.132.0.15:46994 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T01:20:08.951Z] "POST /maas-api/internal/v1/api-keys/cleanup HTTP/1.1" 404 NR route_not_found - "-" 2 0 8 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "f8882e8b-1e1b-4e21-88b2-56e29d251128" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "-" - - 10.133.0.46:443 10.132.0.15:46216 - - 2026-11-04T01:20:09.258705Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:20:09.258739Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:20:09.258751Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T01:20:09.258756Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-04T01:20:09.181Z] "POST /maas-api/internal/v1/api-keys/validate HTTP/1.1" 404 NR route_not_found - "-" 31 0 9 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "faa90641-7bbe-4664-8e18-ce9edfe4822e" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "-" - - 10.133.0.46:443 10.132.0.15:46224 - - [2026-11-04T01:20:09.251Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 91 7 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "df243d46-2da8-4dd8-8c17-af9d72062530" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "10.134.0.21:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.46:58224 10.133.0.46:443 10.132.0.15:46230 - odh-ai-gateway-infra.maas-api-route.4 2026-11-04T01:20:11.063422Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-04T01:20:11.066275Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:20:11.066314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:20:11.066336Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T01:20:11.066343Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-04T01:20:10.873Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 289 76 - "44.212.242.249,10.132.0.15" "-" "68ffc831-6cdc-46ba-87d7-5936846d472b" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "10.132.0.52:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.46:40152 10.133.0.46:443 10.132.0.15:46252 - v1-completions-path [2026-11-04T01:20:11.009Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 316 57 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "c1e08ca2-8f9d-47d2-b535-da67379fc2dd" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "10.134.0.21:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.46:58224 10.133.0.46:443 10.132.0.15:46262 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T01:20:16.915015Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:20:30.094765Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:24:01.113Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 136 369 163 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "c4b4178f-b28f-4c31-9e1a-776f63298898" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "10.132.0.52:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.46:56370 10.133.0.46:443 10.132.0.15:49684 - v1-chat-completions-path [2026-11-04T01:24:28.111Z] "POST /maas-api/v1/tokens HTTP/1.1" 401 - - "-" 20 88 23 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "f2b91864-c3ee-4af0-996d-10048c732b58" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.133.0.46:443 10.132.0.15:60940 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T01:25:16.103887Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:26:06.080187Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:27:24.995111Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:29:23.416044Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:31:27.079675Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:32:03.173511Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:33:26.757377Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:35:02.640179Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:36:04.712793Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:36:04.958231Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:36:04.958274Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:36:04.958293Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:36:04.958301Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T01:36:04.855Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 102 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "a8cf5c51-b2a7-42c0-a349-1c102472afc8" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "10.134.0.21:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.46:43160 10.133.0.46:443 10.132.0.15:60220 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T01:36:27.143064Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:37:11.498148Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-04T01:37:11.498171Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-04T01:37:11.498174Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-04T01:37:11.498178Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-11-04T01:37:11.453Z] "POST /llm/premium-simulated-simulated-premium/v1/completions HTTP/1.1" 403 - - "-" 74 77 44 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "013435cc-a2d8-4de1-a774-c9b5658124ab" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "-" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.46:443 10.132.0.15:59384 - v1-completions-path 2026-11-04T01:37:14.384678Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:37:30.445781Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:37:32.106225Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:37:35.605046Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-04T01:37:35.520Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 85 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "a8f4e241-7777-40b6-933a-4ddcf866d7cd" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "10.133.0.49:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.46:56236 10.133.0.46:443 10.132.0.15:50814 - v1-completions-path [2026-11-04T01:37:35.779Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 63 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "f61a8268-24f3-4757-b3c9-5017889ae896" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.46:443 10.132.0.15:50820 - v1-completions-path 2026-11-04T01:37:37.769599Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:37:48.381451Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:37:51.389277Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:38:00.386704Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:38:04.706Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 66 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "852d59ce-aa92-44a4-bf80-3e4aad54ab79" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "10.133.0.49:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.46:41692 10.133.0.46:443 10.132.0.15:51902 - v1-completions-path [2026-11-04T01:38:04.932Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 61 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "163f8acc-9951-4987-90a2-b6a743c025b5" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "10.133.0.49:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.46:56236 10.133.0.46:443 10.132.0.15:51916 - v1-completions-path [2026-11-04T01:38:05.627Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 53 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "c1574859-c038-4b4b-b1af-3c9fc5ba7721" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.46:443 10.132.0.15:51928 - v1-completions-path [2026-11-04T01:38:07.347Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 59 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "4501b8de-b9a1-45e6-9e2c-0cd33ab3c2a8" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "10.133.0.49:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.46:56236 10.133.0.46:443 10.132.0.15:51934 - v1-completions-path 2026-11-04T01:38:22.923497Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:38:33.056443Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:38:35.282449Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:38:40.528384Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:38:40.528419Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:38:40.528432Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:38:40.528436Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-04T01:38:40.454Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 318 73 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "6fecf303-64a9-47e9-9ab4-ea3d07fcd854" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "10.134.0.21:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.46:43090 10.133.0.46:443 10.132.0.15:40814 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T01:38:53.995895Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:38:55.859841Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:39:01.744016Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:39:13.256519Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:39:26.877120Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:39:28.532259Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:39:34.486955Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:39:52.043407Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:39:54.923617Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:40:10.470823Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:40:12.098611Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:40:15.270Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 76 0 40 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "6962449e-7ca2-41c9-8759-544af7ae5606" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.46:443 10.132.0.15:50794 - v1-completions-path 2026-11-04T01:40:17.925983Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:40:21.836Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 65 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "df9d149e-7bb8-4011-a1e1-f83da225ea5b" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "10.133.0.49:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.46:56236 10.133.0.46:443 10.132.0.15:46438 - v1-completions-path 2026-11-04T01:40:33.548650Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:40:43.786781Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:40:59.421186Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:41:02.160071Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:43:16.979197Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:43:20.124767Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:44:40.044513Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:44:55.594147Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:44:57.382277Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:45:03.748224Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:47:15.668495Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:47:15.668530Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:47:15.668541Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T01:47:15.668545Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-04T01:47:15.614Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 54 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "4029e86e-1ecd-4228-9942-4bea97715035" "maas.apps.db7c6f5a60cadf03a449.prod.konflux-ocp-ci.dev" "10.134.0.21:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.46:60718 10.133.0.46:443 10.132.0.15:54618 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T01:47:22.227648Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:47:35.987020Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:47:51.780855Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:48:03.559590Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:48:07.463883Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:48:15.397521Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:48:17.975745Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:48:33.743051Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:49:11.857036Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-04T01:51:08.033739Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:51:14.053643Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:51:29.844330Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:51:31.515878Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:51:46.433472Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:51:57.272542Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8