2026-11-01T20:06:38.521335Z info FLAG: --concurrency="0" 2026-11-01T20:06:38.521354Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-01T20:06:38.521358Z info FLAG: --help="false" 2026-11-01T20:06:38.521361Z info FLAG: --log_as_json="false" 2026-11-01T20:06:38.521363Z info FLAG: --log_caller="" 2026-11-01T20:06:38.521365Z info FLAG: --log_output_level="default:info" 2026-11-01T20:06:38.521367Z info FLAG: --log_stacktrace_level="default:none" 2026-11-01T20:06:38.521372Z info FLAG: --log_target="[stdout]" 2026-11-01T20:06:38.521374Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-01T20:06:38.521376Z info FLAG: --outlierLogPath="" 2026-11-01T20:06:38.521378Z info FLAG: --profiling="true" 2026-11-01T20:06:38.521380Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-01T20:06:38.521383Z info FLAG: --proxyLogLevel="warning" 2026-11-01T20:06:38.521385Z info FLAG: --serviceCluster="istio-proxy" 2026-11-01T20:06:38.521387Z info FLAG: --stsPort="0" 2026-11-01T20:06:38.521389Z info FLAG: --templateFile="" 2026-11-01T20:06:38.521391Z info FLAG: --tokenManagerPlugin="" 2026-11-01T20:06:38.521394Z info FLAG: --vklog="0" 2026-11-01T20:06:38.521396Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-01T20:06:38.521545Z info Proxy role ips=[10.132.0.70] type=router id=maas-default-gateway-openshift-default-fd4b99879-bnsdt.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-01T20:06:38.521593Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-01T20:06:38.522703Z info cpu limit detected as 2, setting concurrency 2026-11-01T20:06:38.522933Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-01T20:06:38.522945Z info JWT policy is third-party-jwt 2026-11-01T20:06:38.522948Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-01T20:06:38.524888Z info Opening status port 15020 2026-11-01T20:06:38.525237Z info Starting default Istio SDS Server 2026-11-01T20:06:38.525260Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-01T20:06:38.525280Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-01T20:06:38.525959Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-01T20:06:38.527307Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-01T20:06:38.528086Z info Starting proxy agent 2026-11-01T20:06:38.528115Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-01T20:06:38.537266Z info sds Starting SDS grpc server 2026-11-01T20:06:38.537287Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-01T20:06:38.592016Z info cache generated new workload certificate resourceName=default latency=65.285732ms ttl=23h59m59.407987799s 2026-11-01T20:06:38.592053Z info cache Root cert has changed, start rotating root cert 2026-11-01T20:06:38.592083Z info cache returned workload trust anchor from cache ttl=23h59m59.407917114s 2026-11-01T20:06:38.614204Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-01T20:06:38.786621Z info ads ADS: new connection for node:1 2026-11-01T20:06:38.786686Z info cache returned workload certificate from cache ttl=23h59m59.213315614s 2026-11-01T20:06:38.787212Z info ads ADS: new connection for node:2 2026-11-01T20:06:38.787292Z info cache returned workload trust anchor from cache ttl=23h59m59.212709472s 2026-11-01T20:06:38.915511Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:06:47.297676Z info Readiness succeeded in 8.779533156s 2026-11-01T20:06:47.298060Z info Envoy proxy is ready 2026-11-01T20:06:48.128544Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:06:54.619558Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T20:07:09.306Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 281 96 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "184456bb-23eb-434c-9baa-584be8b076be" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.132.0.60:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.70:53840 10.132.0.70:443 10.132.0.21:59710 - v1-completions-path [2026-11-01T20:07:09.749Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 281 68 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "5eebcc61-f88f-41e3-8625-318b4254b934" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.132.0.60:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.70:53856 10.132.0.70:443 10.132.0.21:52136 - v1-completions-path 2026-11-01T20:08:36.283988Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:08:36.284041Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:08:36.284068Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-01T20:08:36.284076Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-01T20:08:36.001Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 283 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "f4e15dd7-f3c2-4d97-80a6-cdcb1ca0ccba" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.70:36302 10.132.0.70:443 10.132.0.21:58938 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T20:09:42.452374Z info ads ADS: new connection for node:3 2026-11-01T20:09:42.458640Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-01T20:09:42.458800Z info cache read certificate from file resource=file-root:system 2026-11-01T20:11:52.708650Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:14:35.931701Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:14:53.561606Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:15:11.298500Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:15:27.046464Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:15:37.138567Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:15:37.138724Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:15:37.138842Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T20:15:37.138885Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:15:37.139232Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-01T20:15:36.887Z] "POST /anything/e2e-dft-75f85b-37e4d3 HTTP/1.1" 200 - via_upstream - "-" 109 1766 252 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "1d3ed6a3-ea0b-4b39-90c4-2114c23a942c" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.70:45680 10.132.0.70:443 10.132.0.21:56022 - llm.external-model-e2e-dft-75f85b-80959e40cd35.0 2026-11-01T20:15:48.440848Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:15:50.937481Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:15:51.492210Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:15:52.694241Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:15:53.762292Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:15:55.884362Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:15:56.595026Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:15:59.602452Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:16:13.360666Z info ads ADS: new connection for node:4 2026-11-01T20:16:13.360919Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-01T20:16:13.360947Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-01T20:17:49.037947Z info ads ADS: "" 4 terminated with stream closed 2026-11-01T20:18:09.938698Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:18:20.030378Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:18:39.615627Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:18:54.527391Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:19:12.420436Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:19:12.420469Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:19:12.420479Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-01T20:19:12.420484Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:19:12.420640Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-01T20:19:12.319Z] "POST /llm/e2e-rt-sw-65ba84/anything/e2e-rt-sw-65ba84-client HTTP/1.1" 503 UH no_healthy_upstream - "-" 124 19 101 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "a85aea26-9c14-40b1-bddd-8a25aac767a2" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rt-sw-65ba84.llm.svc.cluster.local - 10.132.0.70:443 10.132.0.21:50414 - llm.external-model-e2e-rt-sw-65ba84-0337161797d3.2 2026-11-01T20:19:17.596602Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:19:23.757500Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:19:23.757548Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:19:23.757565Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:19:23.757573Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T20:19:23.757917Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-01T20:19:23.530Z] "POST /anything/e2e-rt-sw-65ba84-client HTTP/1.1" 200 - via_upstream - "-" 124 1750 227 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "8d0a8bed-8bba-4082-a88e-c9908c03fc67" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.70:44858 10.132.0.70:443 10.132.0.21:58496 - llm.external-model-e2e-rt-sw-65ba84-0337161797d3.0 2026-11-01T20:19:39.532900Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T20:19:40.073Z] "POST /anything/e2e-rt-eq-1dfd6c-client HTTP/1.1" 200 - via_upstream - "-" 124 1507 133 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "36596405-f7f4-4ef2-af02-39c468c39fc0" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.70:40556 10.132.0.70:443 10.132.0.21:50348 - llm.external-model-e2e-rt-eq-1dfd6c-6b43b9fb9051.2 2026-11-01T20:19:50.470084Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-01T20:19:50.515695Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:19:50.515740Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:19:50.515758Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:19:50.515765Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T20:19:50.516085Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-01T20:19:50.402Z] "POST /anything/e2e-rt-eq-1dfd6c-client HTTP/1.1" 200 - via_upstream - "-" 124 1755 113 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "25a7dd28-be53-4c2f-b829-cd06fd2701cc" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.70:44858 10.132.0.70:443 10.132.0.21:55900 - llm.external-model-e2e-rt-eq-1dfd6c-6b43b9fb9051.0 2026-11-01T20:20:14.337209Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:20:27.534164Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:21:38.595665Z info ads ADS: "" 3 terminated with stream closed 2026-11-01T20:21:51.839809Z info ads ADS: new connection for node:5 2026-11-01T20:21:51.876479Z info cache read certificate from file resource=file-root:system 2026-11-01T20:23:15.784155Z info ads ADS: "" 5 terminated 2026-11-01T20:25:37.343719Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:25:41.378135Z info ads ADS: new connection for node:6 2026-11-01T20:25:41.383580Z info cache read certificate from file resource=file-root:system 2026-11-01T20:26:53.342926Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:27:07.162714Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:27:50.986889Z info ads ADS: "" 6 terminated with stream closed 2026-11-01T20:29:03.152682Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:34:57.865935Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-01T20:35:16.159765Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:35:39.271141Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:37:46.374356Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:38:01.597619Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:38:03.950702Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:38:12.843669Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:38:17.912225Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:38:35.125488Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:38:54.928008Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:39:05.095973Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:39:15.490003Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:39:25.604462Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:39:35.912184Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:39:46.082736Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:39:56.350584Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:40:06.535453Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:40:16.915966Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:40:26.933190Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:41:52.496897Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:41:55.230714Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:41:55.230749Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:41:55.230764Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-01T20:41:55.230769Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-01T20:41:55.145Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 61 304 85 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "8cfb678a-a26e-47a3-a8f0-9f228074f4ec" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.70:47228 10.132.0.70:443 10.132.0.21:51548 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T20:42:08.176180Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:42:09.701730Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:42:15.632226Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:42:30.890630Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:42:32.360289Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:42:46.533854Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:42:57.515078Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:42:59.178441Z info ads ADS: new connection for node:7 2026-11-01T20:42:59.185059Z info cache read certificate from file resource=file-root:system 2026-11-01T20:42:59.349122Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:43:00.329438Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:44:12.058514Z info ads ADS: "" 7 terminated with stream closed 2026-11-01T20:44:12.102112Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:44:12.590460Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:44:15.267259Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:44:17.039979Z info ads ADS: new connection for node:8 2026-11-01T20:44:17.042649Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-01T20:44:18.654096Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:44:59.083285Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:45:03.606937Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-01T20:45:03.536Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 336 101 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "33fdad8a-4f2e-4432-8263-da4376327f82" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.132.1.22:8000" outbound|54321||e2e-prio-pool-4cb35ada-inference-pool-ip-a775089e.llm.svc.cluster.local 10.132.0.70:43822 10.132.0.70:443 10.132.0.21:50556 - v1-model-routing 2026-11-01T20:45:17.617271Z info ads ADS: "" 8 terminated 2026-11-01T20:45:17.670682Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:45:19.298970Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:45:22.309363Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:45:26.596487Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:45:38.395101Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:45:55.226842Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:46:09.597353Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:46:09.597390Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:46:09.597406Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-01T20:46:09.597411Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-01T20:46:09.529Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 80 322 67 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "9b791554-115b-45f2-a889-0b9b876f45cd" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.70:44404 10.132.0.70:443 10.132.0.21:43324 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T20:46:10.820028Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:46:22.484745Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:46:26.274420Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:46:36.972359Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:46:39.492022Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:46:43.250623Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:46:43.250662Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:46:43.250681Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:46:43.250686Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-01T20:46:43.196Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 524 54 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "b9c7f56a-3fde-46e2-b29d-055773ae855d" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.70:44404 10.132.0.70:443 10.132.0.21:55434 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T20:46:54.710971Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:50:54.439486Z info ads ADS: new connection for node:9 2026-11-01T20:50:54.444530Z info cache read certificate from file resource=file-root:system 2026-11-01T20:50:54.943226Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:50:56.362351Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:51:12.692279Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:51:12.692321Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:51:12.692338Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:51:12.692344Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-01T20:51:12.692541Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-01T20:51:12.560Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1707 131 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "55ce0c72-f2a1-4441-ad37-12217895f2b7" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.70:58574 10.132.0.70:443 10.132.0.21:42814 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-01T20:51:23.609325Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:51:23.609363Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:51:23.609377Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T20:51:23.609383Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:51:23.609566Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-01T20:51:23.477Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1710 132 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "bb82ebaa-cab1-426e-b527-500def5e89c4" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.70:36874 10.132.0.70:443 10.132.0.21:33316 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-01T20:51:30.152420Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:51:30.152451Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:51:30.152462Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:51:30.152466Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T20:51:30.152621Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-01T20:51:30.054Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1710 98 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "b98a6934-1643-447e-9818-8c750200a06d" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.70:36874 10.132.0.70:443 10.132.0.21:51464 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-01T20:51:51.857407Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:51:51.857443Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:51:51.857455Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:51:51.857459Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-01T20:51:51.857644Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-01T20:51:51.751Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1710 106 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "82780d4b-1110-4abb-aaa0-7a182308df56" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.70:58574 10.132.0.70:443 10.132.0.21:43582 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-01T20:52:28.590114Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:52:28.710448Z info ads ADS: "" 9 terminated 2026-11-01T20:52:28.992510Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:52:28.992549Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:52:28.992564Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:52:28.992569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T20:52:29.102595Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-01T20:52:29.102623Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-01T20:52:29.102627Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-01T20:52:29.102633Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 2026-11-01T20:52:29.104746Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T20:52:28.918Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 74 317 74 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "d6aff79f-a6b1-48d8-a2bd-f3b3effe1797" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.70:54260 10.132.0.70:443 10.132.0.21:46912 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T20:52:29.052Z] "POST /llm/premium-simulated-simulated-premium/v1/completions HTTP/1.1" 403 - - "-" 74 77 50 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "e5a4aafa-e039-4efb-be9b-28b3943aa830" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "-" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.70:443 10.132.0.21:46928 - v1-completions-path 2026-11-01T20:52:32.388247Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T20:52:36.440Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 71 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "26a340d5-9f92-4308-a9f4-0e3fb43bad52" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.70:42302 10.132.0.70:443 10.132.0.21:35550 - v1-completions-path 2026-11-01T20:52:48.869064Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:52:52.090786Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:53:00.602272Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:53:03.366029Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:53:03.366067Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:53:03.366082Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:53:03.366087Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-01T20:53:03.245Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 323 120 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "fb83cd36-fc59-4f7c-9d8f-8cffe19302bd" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.70:44896 10.132.0.70:443 10.132.0.21:40616 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T20:53:05.697Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 60 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "b4cefccc-741d-4cb5-a708-fd74585d31f7" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.70:58494 10.132.0.70:443 10.132.0.21:40618 - v1-completions-path 2026-11-01T20:53:21.919800Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:53:32.160209Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:53:33.660195Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T20:53:39.381Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 64 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "bffe85fa-ce17-41ee-bb41-02935d79f14a" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.70:42302 10.132.0.70:443 10.132.0.21:54590 - v1-completions-path [2026-11-01T20:53:39.641Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 65 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "e16b93fc-e4e4-4223-a448-d328a8739c87" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.70:42302 10.132.0.70:443 10.132.0.21:54592 - v1-completions-path 2026-11-01T20:53:52.521289Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:53:54.026416Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:53:59.327814Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T20:54:08.468Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 288 63 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "0e56d7e0-85b6-40db-85ee-0443c9c36c68" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.132.0.60:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.70:53840 10.132.0.70:443 10.132.0.21:36050 - v1-completions-path 2026-11-01T20:54:10.233276Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:54:23.883261Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:54:25.296708Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:54:30.769535Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:54:49.033637Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:54:51.561257Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T20:54:52.349Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 1760 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "d19516d8-d0aa-44d4-976a-e8182a62825d" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.132.0.60:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.70:53856 10.132.0.70:443 10.132.0.21:48628 - v1-completions-path 2026-11-01T20:55:05.490910Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:55:06.994677Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:55:08.496117Z info cache event for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt : REMOVE, pushing to proxy 2026-11-01T20:55:12.326972Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:55:27.346838Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:55:37.565455Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:55:52.599299Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:55:55.052033Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:58:07.521620Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:58:09.975567Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:58:12.926768Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:58:27.970921Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:58:37.044859Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:58:41.075233Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:58:45.236695Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:58:45.236731Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:58:45.236746Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:58:45.236751Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-01T20:58:45.164Z] "DELETE /v1/api-keys/05dba915-4a47-4b09-946a-632eb0e15ce5 HTTP/1.1" 200 - via_upstream - "-" 0 369 71 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "27efa99f-f7bf-4a0b-8ee8-1a792b052a5e" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.215:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.70:52668 10.132.0.70:443 10.132.0.21:45244 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T20:58:56.712895Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:59:07.057070Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:59:09.963512Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T20:59:13.827Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 66 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "8d795301-5ed8-4bb6-b24e-08f6b5ce707e" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.70:58494 10.132.0.70:443 10.132.0.21:49740 - v1-completions-path 2026-11-01T20:59:15.102407Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:59:15.102444Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:59:15.102459Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T20:59:15.102464Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-01T20:59:15.040Z] "DELETE /v1/api-keys/5caf1ae3-d695-4bea-9417-c5580ada23f7 HTTP/1.1" 200 - via_upstream - "-" 0 365 62 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "72daebc0-a2eb-4025-b14a-4e42cec113f2" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.215:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.70:52668 10.132.0.70:443 10.132.0.21:49752 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T20:59:26.538730Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:59:28.496708Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T20:59:31.784474Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T20:59:37.243Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 59 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "cca0cee3-038e-457d-81fc-d1d0020fc97a" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.70:58494 10.132.0.70:443 10.132.0.21:39302 - v1-completions-path [2026-11-01T20:59:37.771Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 61 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "cd08f989-d55f-44e9-856f-b77b50ae5d77" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.70:58494 10.132.0.70:443 10.132.0.21:39320 - v1-completions-path [2026-11-01T20:59:38.619Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 55 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "5b40665b-3e5e-4229-a137-bbe368cef7b8" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.70:58494 10.132.0.70:443 10.132.0.21:39334 - v1-completions-path [2026-11-01T20:59:38.829Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 58 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "3a4a4f5a-8d15-469d-a818-b1abd064918d" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.70:58494 10.132.0.70:443 10.132.0.21:39342 - v1-completions-path [2026-11-01T20:59:39.702Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 65 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "1151438f-1969-4b27-9b6c-a4b0ac2cf787" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.70:42302 10.132.0.70:443 10.132.0.21:43918 - v1-completions-path 2026-11-01T20:59:53.287351Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T21:00:03.577147Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T21:00:05.500309Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T21:00:08.835243Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T21:00:14.601295Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T21:00:20.061Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 70 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "2cf9ad10-b894-4dc7-bb02-6ee1094f2b60" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.70:58494 10.132.0.70:443 10.132.0.21:49540 - v1-completions-path 2026-11-01T21:00:33.143507Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T21:00:43.389564Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8