2026-10-30T19:19:33.686225Z info FLAG: --concurrency="0" 2026-10-30T19:19:33.686248Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T19:19:33.686253Z info FLAG: --help="false" 2026-10-30T19:19:33.686255Z info FLAG: --log_as_json="false" 2026-10-30T19:19:33.686257Z info FLAG: --log_caller="" 2026-10-30T19:19:33.686259Z info FLAG: --log_output_level="default:info" 2026-10-30T19:19:33.686261Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T19:19:33.686265Z info FLAG: --log_target="[stdout]" 2026-10-30T19:19:33.686268Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T19:19:33.686269Z info FLAG: --outlierLogPath="" 2026-10-30T19:19:33.686272Z info FLAG: --profiling="true" 2026-10-30T19:19:33.686274Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T19:19:33.686276Z info FLAG: --proxyLogLevel="warning" 2026-10-30T19:19:33.686278Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T19:19:33.686280Z info FLAG: --stsPort="0" 2026-10-30T19:19:33.686282Z info FLAG: --templateFile="" 2026-10-30T19:19:33.686284Z info FLAG: --tokenManagerPlugin="" 2026-10-30T19:19:33.686287Z info FLAG: --vklog="0" 2026-10-30T19:19:33.686289Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T19:19:33.686431Z info Proxy role ips=[10.133.0.39] type=router id=maas-default-gateway-openshift-default-6f54779779-j8c67.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T19:19:33.686480Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T19:19:33.687616Z info cpu limit detected as 2, setting concurrency 2026-10-30T19:19:33.687874Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T19:19:33.687885Z info JWT policy is third-party-jwt 2026-10-30T19:19:33.687888Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T19:19:33.690455Z info Starting default Istio SDS Server 2026-10-30T19:19:33.690483Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T19:19:33.690512Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T19:19:33.690962Z info Opening status port 15020 2026-10-30T19:19:33.691869Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T19:19:33.693340Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T19:19:33.694199Z info Starting proxy agent 2026-10-30T19:19:33.694238Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T19:19:33.697960Z info sds Starting SDS grpc server 2026-10-30T19:19:33.697976Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T19:19:33.762975Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T19:19:33.768910Z info cache generated new workload certificate resourceName=default latency=76.656232ms ttl=23h59m59.231094735s 2026-10-30T19:19:33.768949Z info cache Root cert has changed, start rotating root cert 2026-10-30T19:19:33.768979Z info cache returned workload trust anchor from cache ttl=23h59m59.231021476s 2026-10-30T19:19:33.876617Z info ads ADS: new connection for node:1 2026-10-30T19:19:33.876725Z info cache returned workload certificate from cache ttl=23h59m59.123277512s 2026-10-30T19:19:33.876994Z info ads ADS: new connection for node:2 2026-10-30T19:19:33.877092Z info cache returned workload trust anchor from cache ttl=23h59m59.122909838s 2026-10-30T19:19:33.959421Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:19:41.996408Z info Readiness succeeded in 8.313054466s 2026-10-30T19:19:41.996784Z info Envoy proxy is ready [2026-10-30T19:20:38.079Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 62 - "34.228.250.194,10.132.0.13" "curl/7.76.1" "7423983e-cd69-40c9-8a0b-cc1375f9beb1" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.39:443 10.132.0.13:49202 - v1-model-routing [2026-10-30T19:20:38.174Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 55 - "34.228.250.194,10.132.0.13" "curl/7.76.1" "d11ec68c-5a12-48f0-ac00-aa469cad235f" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.39:443 10.132.0.13:49218 - v1-model-routing 2026-10-30T19:23:35.360086Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T19:23:35.360123Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T19:23:35.360137Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T19:23:35.360142Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T19:23:35.093Z] "DELETE /v1/api-keys/c39a14e1-527c-4ffb-a211-77fca94ef9d1 HTTP/1.1" 200 - via_upstream - "-" 0 321 266 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "548d1392-962e-4905-8e57-cb47ea89a347" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "10.132.0.44:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.39:58474 10.133.0.39:443 10.132.0.13:56658 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T19:23:44.902Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 288 75 - "34.228.250.194,10.132.0.13" "-" "196b2aaa-cb7f-4abc-b5d7-8b8d84ff608d" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "10.132.0.47:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.39:49674 10.133.0.39:443 10.132.0.13:37144 - v1-completions-path 2026-10-30T19:23:50.722966Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T19:23:50.723003Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T19:23:50.723016Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T19:23:50.723021Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T19:23:50.641Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 81 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "c9506f26-54d1-4c8e-8b7d-ea3f25a4669a" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "10.132.0.44:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.39:33462 10.133.0.39:443 10.132.0.13:37154 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T19:23:51.032Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 278 73 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "4a42a797-3027-4413-a60f-03365b0f4014" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "10.132.0.47:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.39:49680 10.133.0.39:443 10.132.0.13:37160 - v1-completions-path 2026-10-30T19:23:54.620626Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:23:59.108130Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T19:23:59.108166Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T19:23:59.108181Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T19:23:59.108185Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T19:23:59.038Z] "DELETE /v1/api-keys/bf285417-7260-4620-8ea8-272ae150e149 HTTP/1.1" 200 - via_upstream - "-" 0 321 69 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "90fa8f39-6eab-4916-8ea9-f9913dfad5ac" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "10.132.0.44:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.39:33462 10.133.0.39:443 10.132.0.13:47326 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T19:24:01.543Z] "POST /maas-api/internal/v1/api-keys/cleanup HTTP/1.1" 404 NR route_not_found - "-" 2 0 8 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "e4709c9a-1de3-42e5-b67d-c2cee41dc1ba" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "-" - - 10.133.0.39:443 10.132.0.13:47330 - - 2026-10-30T19:24:05.181243Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:24:07.803345Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T19:24:07.803428Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T19:24:07.803455Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T19:24:07.803465Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T19:24:07.725Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 77 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "aa8c6b99-c1db-400f-a074-7325334448ce" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "10.132.0.44:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.39:58474 10.133.0.39:443 10.132.0.13:39522 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T19:28:26.744036Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T19:28:26.744078Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T19:28:26.744099Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T19:28:26.744107Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T19:28:26.662Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 322 81 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "199e616e-4727-9200-b1ff-a24401da0f86" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "10.132.0.44:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.39:32954 10.133.0.39:443 10.132.0.13:51272 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T19:29:17.515392Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:29:31.908811Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:31:16.105334Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:31:40.349766Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:32:44.537405Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:33:12.298129Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:35:08.496641Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:39:25.657339Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:40:11.294978Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:47:01.530184Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-10-30T19:50:53.827466Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=3 2026-10-30T20:01:10.302202Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:01:10.302236Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:01:10.302249Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:01:10.302254Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T20:01:10.207Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 69 312 94 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "b73253d0-284d-41e8-922d-511ae9ed89bf" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "10.132.0.44:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.39:38976 10.133.0.39:443 10.132.0.13:48680 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T20:01:10.336Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 110 321 107 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "07b50dac-27bf-4a5f-9f30-f8c2ccf0924e" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "10.132.0.47:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.39:49674 10.133.0.39:443 10.132.0.13:48690 - v1-chat-completions-path 2026-10-30T20:04:42.681132Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:04:42.681166Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:04:42.681179Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:04:42.681184Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T20:04:42.606Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 30 314 74 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "91c80c2b-1f40-450f-98f8-0fce64b0fae0" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "10.132.0.44:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.39:46366 10.133.0.39:443 10.132.0.13:35476 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T20:04:45.817479Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:05:10.271326Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:05:21.495501Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:05:21.495538Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:05:21.495553Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:05:21.495559Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T20:05:21.425Z] "POST /v1/api-keys HTTP/1.1" 400 - via_upstream - "-" 80 94 69 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "2b0d295f-2b31-4625-acff-c032c88e9ffe" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "10.132.0.44:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.39:46366 10.133.0.39:443 10.132.0.13:50306 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T20:05:46.355792Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T20:05:46.355817Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T20:05:46.355820Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T20:05:46.355824Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-10-30T20:05:46.317Z] "POST /llm/premium-simulated-simulated-premium/v1/completions HTTP/1.1" 403 - - "-" 74 77 38 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "85191f0b-a1fa-461d-adbd-4330a9500705" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "-" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.39:443 10.132.0.13:49924 - v1-completions-path 2026-10-30T20:05:51.223550Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:05:55.424021Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:05:55.424060Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:05:55.424078Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:05:55.424084Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T20:05:55.351Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 87 329 72 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "284df100-6c53-4f09-8284-22dcb98eef14" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "10.132.0.44:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.39:46366 10.133.0.39:443 10.132.0.13:51148 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T20:05:56.426931Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-10-30T20:05:56.422Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 59 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "d298904d-3069-4e6d-a145-c85eaa0ff133" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.39:443 10.132.0.13:51150 - v1-completions-path 2026-10-30T20:06:08.426529Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:06:13.456071Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:06:31.597081Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:06:41.354416Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=4 2026-10-30T20:06:47.125241Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T20:07:03.742Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 72 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "35e5193e-05f5-4e7f-bb60-c9c328e42c21" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.39:34220 10.133.0.39:443 10.132.0.13:56338 - v1-completions-path 2026-10-30T20:07:20.756819Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:07:31.183647Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:07:35.257004Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T20:07:41.210Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 59 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "d8fcedce-00b2-4652-ac2a-76ee11fbc361" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.39:34220 10.133.0.39:443 10.132.0.13:57438 - v1-completions-path [2026-10-30T20:07:41.838Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 56 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "41354dc6-b933-4e78-8971-2db5f6e23d0f" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.39:35116 10.133.0.39:443 10.132.0.13:57450 - v1-completions-path 2026-10-30T20:07:53.732955Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:07:56.003047Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:08:15.188362Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T20:08:17.222Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 290 67 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "7f6c427a-aa78-4436-b8c5-0e710ed2dd07" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "10.132.0.47:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.39:49674 10.133.0.39:443 10.132.0.13:60174 - v1-completions-path 2026-10-30T20:08:19.808145Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:08:36.347188Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:08:38.476434Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:08:57.474934Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:09:08.301312Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:09:11.172031Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:09:30.416997Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:09:32.992997Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:10:04.379813Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:10:06.985706Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:11:54.624659Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:11:57.834862Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:13:27.135610Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:13:42.209442Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:13:44.204265Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:13:59.118682Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:16:23.629326Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:16:23.629360Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:16:23.629372Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T20:16:23.629377Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T20:16:23.561Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 67 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "7502f8f1-3e4d-443e-90ea-4c97672e209f" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "10.132.0.44:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.39:42758 10.133.0.39:443 10.132.0.13:49130 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T20:16:31.409981Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:17:00.227744Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:17:34.333784Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:17:34.333821Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:17:34.333839Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T20:17:34.333843Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T20:17:34.242Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 80 323 91 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "92aca235-1bfc-4d31-8d6e-58a0e39d8cb0" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "10.132.0.44:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.39:47828 10.133.0.39:443 10.132.0.13:45906 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T20:17:36.165452Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:17:47.240975Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:17:58.371748Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:18:03.377903Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:18:06.495186Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:18:17.240871Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:18:21.109273Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T20:18:25.485Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 58 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "0d961b8d-7ac5-4ded-a38e-1813c0e5c78b" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.39:34220 10.133.0.39:443 10.132.0.13:59980 - v1-completions-path 2026-10-30T20:18:37.510332Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:19:17.394793Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:19:22.621329Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:19:25.501022Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-10-30T20:19:25.400Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 100 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "075eb71a-8f25-40c1-8f74-61e55f79181e" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "10.134.0.35:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.39:56518 10.133.0.39:443 10.132.0.13:40384 - v1-catch-all-path 2026-10-30T20:19:37.871113Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:19:39.874699Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:19:42.662182Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-10-30T20:19:42.599Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 62 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "15f2a44b-3db3-410e-b0c9-b57acc1ebf7d" "maas.apps.88a23b71f093755d6258.prod.konflux-ocp-ci.dev" "10.134.0.35:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.39:56518 10.133.0.39:443 10.132.0.13:53026 - v1-catch-all-path 2026-10-30T20:19:54.463144Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:20:05.352778Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8