2026-10-30T20:52:37.224578Z info FLAG: --concurrency="0" 2026-10-30T20:52:37.227622Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T20:52:37.227753Z info FLAG: --help="false" 2026-10-30T20:52:37.227784Z info FLAG: --log_as_json="false" 2026-10-30T20:52:37.227805Z info FLAG: --log_caller="" 2026-10-30T20:52:37.227826Z info FLAG: --log_output_level="default:info" 2026-10-30T20:52:37.227845Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T20:52:37.227874Z info FLAG: --log_target="[stdout]" 2026-10-30T20:52:37.227895Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T20:52:37.227914Z info FLAG: --outlierLogPath="" 2026-10-30T20:52:37.227935Z info FLAG: --profiling="true" 2026-10-30T20:52:37.227957Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T20:52:37.227976Z info FLAG: --proxyLogLevel="warning" 2026-10-30T20:52:37.227996Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T20:52:37.228019Z info FLAG: --stsPort="0" 2026-10-30T20:52:37.228041Z info FLAG: --templateFile="" 2026-10-30T20:52:37.228061Z info FLAG: --tokenManagerPlugin="" 2026-10-30T20:52:37.228083Z info FLAG: --vklog="0" 2026-10-30T20:52:37.228113Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T20:52:37.228369Z info Proxy role ips=[10.134.0.64] type=router id=maas-default-gateway-openshift-default-6db586dd6c-npbcs.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T20:52:37.228480Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T20:52:37.230990Z info cpu limit detected as 2, setting concurrency 2026-10-30T20:52:37.231477Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T20:52:37.231555Z info JWT policy is third-party-jwt 2026-10-30T20:52:37.231582Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T20:52:37.236774Z info Starting default Istio SDS Server 2026-10-30T20:52:37.237060Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T20:52:37.237125Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T20:52:37.236858Z info Opening status port 15020 2026-10-30T20:52:37.238247Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T20:52:37.255725Z info sds Starting SDS grpc server 2026-10-30T20:52:37.255766Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T20:52:37.257012Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T20:52:37.258437Z info Starting proxy agent 2026-10-30T20:52:37.258549Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T20:52:37.395718Z info cache generated new workload certificate resourceName=default latency=145.616071ms ttl=23h59m59.604286862s 2026-10-30T20:52:37.395777Z info cache Root cert has changed, start rotating root cert 2026-10-30T20:52:37.395823Z info cache returned workload trust anchor from cache ttl=23h59m59.604178157s 2026-10-30T20:52:37.435409Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T20:52:37.712750Z info ads ADS: new connection for node:1 2026-10-30T20:52:37.714206Z info cache returned workload certificate from cache ttl=23h59m59.285797742s 2026-10-30T20:52:37.719024Z info ads ADS: new connection for node:2 2026-10-30T20:52:37.719213Z info cache returned workload trust anchor from cache ttl=23h59m59.280789687s 2026-10-30T20:52:37.977999Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T20:52:45.746051Z error failed scraping envoy metrics: error scraping http://localhost:15090/stats/prometheus: Get "http://localhost:15090/stats/prometheus": dial tcp [::1]:15090: connect: connection refused 2026-10-30T20:52:47.913844Z info Readiness succeeded in 10.697144409s 2026-10-30T20:52:47.914424Z info Envoy proxy is ready 2026-10-30T20:55:37.565504Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:55:37.565695Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:55:37.565769Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:55:37.565802Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T20:55:37.388Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 177 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "94eb3b8d-f5f3-4e66-a5f6-6cc41f6fe379" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.58:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.64:43460 10.134.0.64:443 10.132.0.10:56544 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T20:56:15.006965Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T20:56:25.751132Z error failed scraping envoy metrics: error scraping http://localhost:15090/stats/prometheus: Get "http://localhost:15090/stats/prometheus": context deadline exceeded 2026-10-30T20:56:25.875311Z warn xdsproxy sendDownstreamDelta took 10.848877154s 2026-10-30T20:56:45.313788Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T20:57:23.924144Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T20:58:28.422493Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:03:04.424417Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:03:26.663466Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:04:14.200040Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T21:04:14.200071Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T21:04:14.200076Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T21:04:14.200081Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-10-30T21:04:14.155Z] "POST /llm/premium-simulated-simulated-premium/v1/completions HTTP/1.1" 403 - - "-" 74 77 44 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "a50067da-df2d-4a5e-aed2-37fc6cdbe9b9" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "-" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.64:443 10.132.0.10:54846 - v1-completions-path 2026-10-30T21:04:17.209704Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-30T21:04:23.371Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 74 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "1a7f6fbb-5f71-4b2d-86f3-bbc04e09125a" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.63:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.64:57534 10.134.0.64:443 10.132.0.10:33382 - v1-completions-path [2026-10-30T21:04:23.607Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 69 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "79edf1e4-d343-40c1-9678-f2ba7c85101b" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.63:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.64:57544 10.134.0.64:443 10.132.0.10:33396 - v1-completions-path [2026-10-30T21:04:23.832Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 60 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "bedaad0a-7da5-4840-ba8e-515032ca9054" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.64:443 10.132.0.10:33404 - v1-completions-path 2026-10-30T21:04:35.579370Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:04:37.455774Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-30T21:04:42.310Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 67 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "457a3c73-5d9c-46b8-8575-16333c8edc13" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.63:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.64:57534 10.134.0.64:443 10.132.0.10:47192 - v1-completions-path [2026-10-30T21:04:42.556Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 61 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "955379f1-8a7f-4f03-a4aa-3d726b35c5d3" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.64:443 10.132.0.10:47200 - v1-completions-path 2026-10-30T21:04:54.261056Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:04:56.141770Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:04:58.922733Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:05:06.556324Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:05:16.874483Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:05:20.831133Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-10-30T21:05:20.773Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 62 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "779db8e3-4ce9-4f78-a770-a078e00e08c0" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.63:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.64:57534 10.134.0.64:443 10.132.0.10:51566 - v1-completions-path [2026-10-30T21:05:21.335Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 297 55 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "5589b04e-35a2-40e4-b18f-7781ef1ef22c" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.63:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.64:57534 10.134.0.64:443 10.132.0.10:51572 - v1-completions-path [2026-10-30T21:05:21.576Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 57 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "23156bd1-fcc6-4ddf-973e-7f49339958a1" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.63:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.64:57534 10.134.0.64:443 10.132.0.10:51588 - v1-completions-path [2026-10-30T21:05:23.739Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 298 56 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "355a10e1-70da-44f7-8fe2-5ab3cbbd5c6f" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.63:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.64:57534 10.134.0.64:443 10.132.0.10:51592 - v1-completions-path 2026-10-30T21:05:38.828072Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:05:48.937648Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:05:50.732960Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-30T21:05:56.800Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 64 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "50f0832b-c415-4f0e-b84e-aacb63c8926b" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.63:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.64:57534 10.134.0.64:443 10.132.0.10:33178 - v1-completions-path [2026-10-30T21:05:57.942Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 94 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "16542700-b1b6-4fee-8ff4-d5225fce01c5" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.63:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.64:57534 10.134.0.64:443 10.132.0.10:33184 - v1-completions-path 2026-10-30T21:06:09.879285Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:06:11.644773Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-30T21:06:16.489Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 74 295 80 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "6215d671-e1c9-4350-baa2-b24238783bd0" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.60:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.134.0.64:48538 10.134.0.64:443 10.132.0.10:46628 - v1-completions-path 2026-10-30T21:06:28.170717Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:06:30.437573Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:06:46.939569Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:06:48.789922Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:07:14.792117Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:07:25.463314Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:07:28.153199Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:07:47.463918Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:07:49.690669Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:08:02.950968Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T21:08:02.951010Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T21:08:02.951031Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T21:08:02.951036Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T21:08:02.876Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 89 331 74 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "47396e31-1eca-4297-b65f-6c71ed41d0c2" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.58:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.64:47494 10.134.0.64:443 10.132.0.10:58330 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T21:08:04.766Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 76 0 34 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "b30e8669-37a5-4df4-83e5-feeb1724f105" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.64:443 10.132.0.10:58348 - v1-completions-path 2026-10-30T21:08:07.373241Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:08:25.875791Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:08:36.084014Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:09:02.611345Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:09:05.500968Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:10:40.466626Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:10:43.526043Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:12:20.746174Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:12:23.417944Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T21:12:23.417987Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T21:12:23.418005Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T21:12:23.418011Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T21:12:23.247Z] "DELETE /v1/api-keys/e2e-trlp-test-key HTTP/1.1" 404 - via_upstream - "-" 0 29 170 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "a42681f3-8492-4406-b2cc-091cc04a8e50" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.58:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.64:59408 10.134.0.64:443 10.132.0.10:38340 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T21:12:34.958728Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:12:46.220206Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:13:01.014232Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:15:02.240396Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T21:15:02.240444Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T21:15:02.240467Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-10-30T21:15:02.240473Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T21:15:02.240481Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=26 [2026-10-30T21:15:02.174Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 65 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "db349929-d727-449c-97ec-235021ddafc3" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.58:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.64:51562 10.134.0.64:443 10.132.0.10:42916 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T21:15:07.276199Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:15:24.802423Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:15:39.792309Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:15:48.618027Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T21:15:48.618077Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T21:15:48.618097Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T21:15:48.618103Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T21:15:48.554Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 80 323 63 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "5ecf8c0a-74e6-4e19-ae9d-8a57c3898309" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.58:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.64:40158 10.134.0.64:443 10.132.0.10:53394 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T21:16:01.728859Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:16:05.613165Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:16:06.245995Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T21:16:06.246046Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T21:16:06.246069Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T21:16:06.246078Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-10-30T21:16:06.149Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 96 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "b9cd030b-c7c3-4d45-9b39-278fb29e0014" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.58:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.64:35456 10.134.0.64:443 10.132.0.10:39828 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T21:16:17.716765Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:16:28.845181Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:16:46.048219Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:19:50.228309Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:19:55.437621Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:19:58.347522Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T21:19:58.347561Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T21:19:58.347576Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T21:19:58.347582Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T21:19:58.944611Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-10-30T21:19:58.188Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 317 158 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "450f02bd-3952-4287-9935-0a471e240c2f" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.133.0.139:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.64:51782 10.134.0.64:443 10.132.0.10:46232 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T21:19:58.869Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 75 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "b067e4ea-c99a-44dc-8321-5ed332a0c7a0" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.62:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.64:46796 10.134.0.64:443 10.132.0.10:46248 - v1-catch-all-path 2026-10-30T21:20:10.572886Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:20:12.289407Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:20:14.974516Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T21:20:14.974566Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T21:20:14.974590Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T21:20:14.974597Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-10-30T21:20:14.853Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 73 316 121 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "5e23b19a-91f3-4a13-bfae-b3ab5fa14940" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.130:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.64:34854 10.134.0.64:443 10.132.0.10:40116 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T21:20:26.631895Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T21:20:37.346699Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9