2026-10-30T05:43:18.483882Z info FLAG: --concurrency="0" 2026-10-30T05:43:18.483911Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T05:43:18.483915Z info FLAG: --help="false" 2026-10-30T05:43:18.483917Z info FLAG: --log_as_json="false" 2026-10-30T05:43:18.483919Z info FLAG: --log_caller="" 2026-10-30T05:43:18.483921Z info FLAG: --log_output_level="default:info" 2026-10-30T05:43:18.483923Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T05:43:18.483928Z info FLAG: --log_target="[stdout]" 2026-10-30T05:43:18.483930Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T05:43:18.483932Z info FLAG: --outlierLogPath="" 2026-10-30T05:43:18.483934Z info FLAG: --profiling="true" 2026-10-30T05:43:18.483936Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T05:43:18.483938Z info FLAG: --proxyLogLevel="warning" 2026-10-30T05:43:18.483940Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T05:43:18.483942Z info FLAG: --stsPort="0" 2026-10-30T05:43:18.483944Z info FLAG: --templateFile="" 2026-10-30T05:43:18.483946Z info FLAG: --tokenManagerPlugin="" 2026-10-30T05:43:18.483949Z info FLAG: --vklog="0" 2026-10-30T05:43:18.483951Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T05:43:18.484092Z info Proxy role ips=[10.134.0.32] type=router id=maas-default-gateway-openshift-default-84587756dc-xp2kr.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T05:43:18.484138Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T05:43:18.485120Z info cpu limit detected as 2, setting concurrency 2026-10-30T05:43:18.485354Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T05:43:18.485366Z info JWT policy is third-party-jwt 2026-10-30T05:43:18.485368Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T05:43:18.486993Z info Starting default Istio SDS Server 2026-10-30T05:43:18.487014Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T05:43:18.487031Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T05:43:18.487316Z info Opening status port 15020 2026-10-30T05:43:18.488109Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T05:43:18.489433Z info sds Starting SDS grpc server 2026-10-30T05:43:18.489453Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T05:43:18.489651Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T05:43:18.490462Z info Starting proxy agent 2026-10-30T05:43:18.490490Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T05:43:18.552557Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T05:43:18.658777Z info ads ADS: new connection for node:1 2026-10-30T05:43:18.659181Z info ads ADS: new connection for node:2 2026-10-30T05:43:18.709373Z info cache generated new workload certificate resourceName=default latency=219.770661ms ttl=23h59m59.290629715s 2026-10-30T05:43:18.709400Z info cache Root cert has changed, start rotating root cert 2026-10-30T05:43:18.709428Z info cache returned workload certificate from cache ttl=23h59m59.290577432s 2026-10-30T05:43:18.709439Z info cache returned workload trust anchor from cache ttl=23h59m59.290561505s 2026-10-30T05:43:18.709550Z info cache returned workload trust anchor from cache ttl=23h59m59.290450313s 2026-10-30T05:43:18.709809Z info cache returned workload trust anchor from cache ttl=23h59m59.290190953s 2026-10-30T05:43:18.788108Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:43:26.627684Z info Readiness succeeded in 8.145976948s 2026-10-30T05:43:26.628085Z info Envoy proxy is ready [2026-10-30T05:43:40.909Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 61 - "34.228.250.194,10.132.0.20" "curl/7.76.1" "9f182514-c375-496e-8060-938f29cb41af" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.32:443 10.132.0.20:42138 - v1-model-routing 2026-10-30T05:46:17.739969Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T05:46:17.740007Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T05:46:17.740021Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T05:46:17.740027Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T05:46:17.624Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 84 327 115 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "ccdff7a2-2963-476e-a6fc-01007646fd20" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.32:45082 10.134.0.32:443 10.132.0.20:32826 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T05:46:19.892824Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T05:46:19.892858Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T05:46:19.892872Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T05:46:19.892876Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T05:46:20.231922Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T05:46:20.231974Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T05:46:20.231996Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T05:46:20.232003Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T05:46:19.773Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 119 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "ca259b25-4cc7-44e4-89d3-e4f1115576f4" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.32:45092 10.134.0.32:443 10.132.0.20:32836 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T05:46:20.127Z] "DELETE /v1/api-keys/81ac6d53-7bec-4ca9-a458-39014b5446db HTTP/1.1" 200 - via_upstream - "-" 0 321 104 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "ca851788-f22c-469a-8e1e-b677e82a3f81" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.32:45092 10.134.0.32:443 10.132.0.20:32850 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T05:46:20.793596Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T05:46:20.793620Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T05:46:20.793624Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T05:46:20.793630Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 2026-10-30T05:46:21.085660Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T05:46:21.085699Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T05:46:21.085718Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T05:46:21.085725Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T05:46:20.725Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 77 67 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "dd914c91-44de-4bfe-97b0-0b6448f85d7a" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.32:443 10.132.0.20:32854 - v1-completions-path [2026-10-30T05:46:20.993Z] "DELETE /v1/api-keys/31287248-fd66-4ef2-b909-149c4ea1721b HTTP/1.1" 200 - via_upstream - "-" 0 321 92 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "e778c788-a35e-4cb6-83c2-2ddfbfaed0d1" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.32:45092 10.134.0.32:443 10.132.0.20:32866 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T05:46:27.232051Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T05:46:27.232089Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T05:46:27.232105Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T05:46:27.232110Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T05:46:26.935Z] "POST /maas-api/internal/v1/api-keys/validate HTTP/1.1" 404 NR route_not_found - "-" 31 0 9 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "ae9a4202-afdc-4a64-94aa-a5316f01b120" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "-" - - 10.134.0.32:443 10.132.0.20:49228 - - [2026-10-30T05:46:27.157Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 568 75 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "414048f2-fd5e-4483-b7b8-6e4a563ebb94" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.32:45082 10.134.0.32:443 10.132.0.20:49232 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T05:46:42.127223Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:46:56.565019Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:52:40.771740Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:53:40.677694Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:54:20.439403Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:56:09.287392Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:11:28.535767Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-10-30T06:21:02.062043Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:21:02.062083Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:21:02.062099Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:21:02.062105Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T06:21:01.939Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 69 311 122 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "b3e9e479-3ca6-93d9-8cda-7d67c430e124" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.32:42752 10.134.0.32:443 10.132.0.20:50400 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T06:21:02.192Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 110 327 82 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "f7b533b2-0716-47b6-a52a-4bb51d3d4a08" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.32:35524 10.134.0.32:443 10.132.0.20:50404 - v1-chat-completions-path 2026-10-30T06:24:47.934946Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:24:47.934980Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:24:47.934994Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:24:47.934999Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T06:24:47.859Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 30 315 75 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "8c3ccf11-b0d8-4bdd-8d14-92d08ca6e931" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.32:51204 10.134.0.32:443 10.132.0.20:43866 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T06:24:50.708978Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:24:52.788091Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:24:52.788125Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:24:52.788139Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:24:52.788144Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T06:24:52.716Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 71 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "99877f34-dcd7-4044-9204-63587eb57a4a" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.32:54426 10.134.0.32:443 10.132.0.20:43890 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T06:25:15.949838Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:25:27.392624Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:25:27.392661Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:25:27.392678Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:25:27.392685Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T06:25:27.297Z] "POST /v1/api-keys HTTP/1.1" 400 - via_upstream - "-" 80 94 94 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "e2401925-7791-479a-a1f5-ad3138a31a59" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.32:54426 10.134.0.32:443 10.132.0.20:53826 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T06:25:51.914765Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T06:25:51.914787Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T06:25:51.914790Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T06:25:51.914794Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-10-30T06:25:51.879Z] "POST /llm/premium-simulated-simulated-premium/v1/completions HTTP/1.1" 403 - - "-" 74 77 34 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "dd0880c5-3ced-492c-96e7-a98637c9feac" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "-" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.32:443 10.132.0.20:48842 - v1-completions-path 2026-10-30T06:25:56.784306Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:26:13.904342Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:26:17.953959Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:26:21.576831Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:26:21.576871Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:26:21.576890Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:26:21.576898Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T06:26:21.506Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 335 70 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "fe07e413-caa8-9b44-bb63-aa052a4f4efe" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.32:54426 10.134.0.32:443 10.132.0.20:32982 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T06:26:33.080Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 278 66 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "9b67b1f1-cd02-4068-9345-478c0bd6f4ee" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.32:44670 10.134.0.32:443 10.132.0.20:44384 - v1-completions-path 2026-10-30T06:26:33.952864Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:26:36.343626Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:26:39.342266Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:26:49.633301Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:27:12.639855Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:27:23.163775Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:27:27.224595Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T06:27:32.088Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 75 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "fbf371d4-6ce2-482f-8161-d166f96f4983" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.32:38000 10.134.0.32:443 10.132.0.20:50840 - v1-completions-path 2026-10-30T06:27:44.683117Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:27:46.775096Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:27:52.097259Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=3 2026-10-30T06:28:06.228996Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:28:10.669425Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:28:26.296863Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:28:28.398643Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:28:46.436381Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:28:57.331739Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:29:00.435778Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:29:18.292035Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:29:20.857372Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:29:26.635481Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-10-30T06:29:26.579Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 279 60 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "e1c1e568-2529-4dcb-93d6-1d66770d6c23" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.32:44670 10.134.0.32:443 10.132.0.20:42142 - v1-completions-path 2026-10-30T06:29:41.927928Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:29:44.584590Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:31:07.903041Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:31:07.903088Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:31:07.903109Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:31:07.903116Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T06:31:07.813Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 80 322 89 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "d6e5baa6-a194-4c49-bfa0-677f95a331c0" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.32:51904 10.134.0.32:443 10.132.0.20:50054 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T06:31:20.661449Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:31:24.008222Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:32:35.224123Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T06:32:35.224149Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T06:32:35.224153Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T06:32:35.224157Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-10-30T06:32:35.177Z] "POST /llm/e2e-trlp-test-simulated/v1/completions HTTP/1.1" 403 - - "-" 73 113 46 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "139dc8e5-0f7e-49ed-8b99-faf19c9b6de5" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-trlp-test-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.32:443 10.132.0.20:33542 - v1-completions-path 2026-10-30T06:32:54.430986Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:33:09.965577Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:33:12.045686Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:33:27.080302Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:34:39.578379Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:34:39.578405Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-10-30T06:34:39.578418Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:34:39.578432Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:34:39.578437Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T06:34:39.526Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 52 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "90db802e-9ca5-425b-bbaf-f7921cf0276b" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.32:50054 10.134.0.32:443 10.132.0.20:55374 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T06:34:44.996600Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:35:00.369070Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:35:16.377824Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:35:30.600160Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:35:33.639923Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:35:46.383656Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:35:49.983484Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:36:09.819122Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:36:19.542151Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:36:27.684440Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:36:33.541965Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-10-30T06:36:33.465Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 76 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "9d175b4a-812f-4875-b2e0-da63f935008f" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.53:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.32:44172 10.134.0.32:443 10.132.0.20:51780 - v1-catch-all-path 2026-10-30T06:36:41.635663Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:36:43.687634Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:36:58.463300Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:37:09.509156Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8