2026-11-03T13:20:47.265111Z info FLAG: --concurrency="0" 2026-11-03T13:20:47.265128Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T13:20:47.265132Z info FLAG: --help="false" 2026-11-03T13:20:47.265135Z info FLAG: --log_as_json="false" 2026-11-03T13:20:47.265137Z info FLAG: --log_caller="" 2026-11-03T13:20:47.265139Z info FLAG: --log_output_level="default:info" 2026-11-03T13:20:47.265141Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T13:20:47.265145Z info FLAG: --log_target="[stdout]" 2026-11-03T13:20:47.265147Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T13:20:47.265149Z info FLAG: --outlierLogPath="" 2026-11-03T13:20:47.265151Z info FLAG: --profiling="true" 2026-11-03T13:20:47.265153Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T13:20:47.265155Z info FLAG: --proxyLogLevel="warning" 2026-11-03T13:20:47.265157Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T13:20:47.265159Z info FLAG: --stsPort="0" 2026-11-03T13:20:47.265162Z info FLAG: --templateFile="" 2026-11-03T13:20:47.265163Z info FLAG: --tokenManagerPlugin="" 2026-11-03T13:20:47.265166Z info FLAG: --vklog="0" 2026-11-03T13:20:47.265168Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T13:20:47.265299Z info Proxy role ips=[10.134.0.120] type=router id=maas-default-gateway-openshift-default-6cc458d887-9fn86.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T13:20:47.265345Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T13:20:47.266830Z info cpu limit detected as 2, setting concurrency 2026-11-03T13:20:47.267431Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T13:20:47.267443Z info JWT policy is third-party-jwt 2026-11-03T13:20:47.267446Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T13:20:47.269181Z info Starting default Istio SDS Server 2026-11-03T13:20:47.269208Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T13:20:47.269234Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T13:20:47.269876Z info Opening status port 15020 2026-11-03T13:20:47.271196Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T13:20:47.271713Z info sds Starting SDS grpc server 2026-11-03T13:20:47.271819Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T13:20:47.273018Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T13:20:47.274182Z info Starting proxy agent 2026-11-03T13:20:47.274222Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T13:20:47.349069Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T13:20:47.404283Z info cache generated new workload certificate resourceName=default latency=132.368152ms ttl=23h59m59.595720706s 2026-11-03T13:20:47.404376Z info cache Root cert has changed, start rotating root cert 2026-11-03T13:20:47.404461Z info cache returned workload trust anchor from cache ttl=23h59m59.595539549s 2026-11-03T13:20:47.544338Z info ads ADS: new connection for node:1 2026-11-03T13:20:47.549819Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-03T13:20:47.549877Z info cache read certificate from file resource=file-root:system 2026-11-03T13:20:47.576056Z info ads ADS: new connection for node:2 2026-11-03T13:20:47.576108Z info cache returned workload certificate from cache ttl=23h59m59.423893892s 2026-11-03T13:20:47.576970Z info ads ADS: new connection for node:3 2026-11-03T13:20:47.577025Z info cache returned workload trust anchor from cache ttl=23h59m59.422976123s 2026-11-03T13:20:47.684271Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:20:52.192366Z error failed scraping envoy metrics: error scraping http://localhost:15090/stats/prometheus: Get "http://localhost:15090/stats/prometheus": dial tcp [::1]:15090: connect: connection refused 2026-11-03T13:20:58.101149Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:21:01.009555Z warn Envoy proxy is NOT ready: failed to get readiness stats: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(server\\.state|listener_manager\\.workers_started)": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T13:21:01.421368Z info Readiness succeeded in 14.158819295s 2026-11-03T13:21:01.421874Z info Envoy proxy is ready 2026-11-03T13:21:16.609125Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:21:40.741095Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:21:47.006145Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=27 2026-11-03T13:21:47.006377Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=27 [2026-11-03T13:21:46.788Z] "POST /anything/e2e-rt-eq-c2c8a5-client HTTP/1.1" 200 - via_upstream - "-" 124 1757 217 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "c88063e0-c931-4ea5-adc8-81091098faac" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.120:42914 10.134.0.120:443 10.132.0.19:48202 - llm.external-model-e2e-rt-eq-c2c8a5-fabe2c1a5231.2 2026-11-03T13:21:47.811037Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-03T13:21:47.811282Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 2026-11-03T13:21:48.194203Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=27 2026-11-03T13:21:48.194573Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=27 [2026-11-03T13:21:47.613Z] "POST /anything/e2e-rt-eq-c2c8a5-client HTTP/1.1" 200 - via_upstream - "-" 124 1766 197 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "3eeb0fdb-2c0e-4e6a-a75a-93e9e81c8303" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.120:42920 10.134.0.120:443 10.132.0.19:48212 - llm.external-model-e2e-rt-eq-c2c8a5-fabe2c1a5231.2 [2026-11-03T13:21:48.000Z] "POST /anything/e2e-rt-eq-c2c8a5-client HTTP/1.1" 200 - via_upstream - "-" 124 1768 194 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "70e5147d-88b0-4ac8-9536-6945ebbc0bd0" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.120:42928 10.134.0.120:443 10.132.0.19:48224 - llm.external-model-e2e-rt-eq-c2c8a5-fabe2c1a5231.0 2026-11-03T13:21:52.398753Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=27 2026-11-03T13:21:52.398982Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=27 [2026-11-03T13:21:52.199Z] "POST /anything/e2e-rt-eq-c2c8a5-client HTTP/1.1" 200 - via_upstream - "-" 124 1769 199 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "b3f73682-45a3-4837-8b07-e9d4465ad3ab" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.120:57652 10.134.0.120:443 10.132.0.19:48250 - llm.external-model-e2e-rt-eq-c2c8a5-fabe2c1a5231.2 2026-11-03T13:22:14.546330Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:22:21.771778Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:22:35.478063Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:22:41.976707Z info ads ADS: "" 1 terminated 2026-11-03T13:22:55.762193Z info ads ADS: new connection for node:4 2026-11-03T13:22:55.782733Z info cache read certificate from file resource=file-root:system 2026-11-03T13:24:32.775333Z info ads ADS: "" 4 terminated 2026-11-03T13:24:35.425204Z info ads ADS: new connection for node:5 2026-11-03T13:24:35.440631Z info cache read certificate from file resource=file-root:system 2026-11-03T13:25:50.434441Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:26:07.782692Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:26:24.434679Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:28:06.127342Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:29:02.826979Z info ads ADS: "" 5 terminated with stream closed 2026-11-03T13:29:03.709512Z info ads ADS: new connection for node:6 2026-11-03T13:29:03.724705Z info cache read certificate from file resource=file-root:system 2026-11-03T13:29:04.380623Z info ads ADS: "" 6 terminated with stream closed 2026-11-03T13:29:19.075053Z info ads ADS: new connection for node:7 2026-11-03T13:29:19.082624Z info cache read certificate from file resource=file-root:system 2026-11-03T13:29:26.310863Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:29:28.707472Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:29:41.894251Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:29:50.830430Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=27 [2026-11-03T13:29:50.657Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 66 308 173 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "6c15c481-3974-48cb-bcf6-363101394426" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.132.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.120:34898 10.134.0.120:443 10.132.0.19:48704 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T13:29:55.847519Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-03T13:29:55.847954Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-11-03T13:29:52.615Z] "POST /anything/e2e-dft-677ab7-cf2d2a HTTP/1.1" 200 - via_upstream - "-" 109 1782 3232 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "220468da-845b-4a57-b73f-57b88fcc3f5a" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.120:34416 10.134.0.120:443 10.132.0.19:48718 - llm.external-model-e2e-dft-677ab7-88c48b9b3b00.0 2026-11-03T13:30:08.251392Z info ads ADS: "" 7 terminated with stream closed 2026-11-03T13:30:09.264160Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:30:11.353465Z info ads ADS: new connection for node:8 2026-11-03T13:30:11.358420Z info cache read certificate from file resource=file-root:system 2026-11-03T13:30:15.715757Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:30:17.491177Z info ads ADS: "" 8 terminated with stream closed 2026-11-03T13:30:18.068973Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:30:22.067844Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:30:23.997977Z info ads ADS: new connection for node:9 2026-11-03T13:30:24.008909Z info cache read certificate from file resource=file-root:system 2026-11-03T13:30:26.535138Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:30:30.352304Z info ads ADS: "" 9 terminated with stream closed 2026-11-03T13:30:31.401913Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:30:33.073210Z info ads ADS: new connection for node:10 2026-11-03T13:30:33.078862Z info cache read certificate from file resource=file-root:system 2026-11-03T13:30:37.811158Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:30:39.442620Z info ads ADS: "" 10 terminated 2026-11-03T13:30:41.519189Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:30:44.190984Z info ads ADS: new connection for node:11 2026-11-03T13:30:44.201356Z info cache read certificate from file resource=file-root:system 2026-11-03T13:30:49.768025Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:32:45.228023Z info ads ADS: "" 11 terminated 2026-11-03T13:32:55.151163Z info ads ADS: new connection for node:12 2026-11-03T13:32:55.164581Z info cache read certificate from file resource=file-root:system 2026-11-03T13:35:23.986029Z info ads ADS: new connection for node:13 2026-11-03T13:35:23.986260Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T13:35:23.986318Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T13:36:32.807909Z info ads ADS: "" 13 terminated 2026-11-03T13:37:32.050424Z info ads ADS: "" 12 terminated 2026-11-03T13:39:30.347208Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-03T13:39:30.160Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 30 314 186 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "aeec0ba9-67ad-4aca-bf8d-1f3157710b94" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.132.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.120:42806 10.134.0.120:443 10.132.0.19:51388 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T13:39:38.081943Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:40:01.670114Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:40:37.760772Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:40:57.372089Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:41:02.506123Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:41:14.898484Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:41:29.264989Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:41:39.229948Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:41:40.300947Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-03T13:41:40.130Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 73 316 170 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "32505f75-640a-4612-b6f4-ffee18c5e600" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.132.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.120:37932 10.134.0.120:443 10.132.0.19:57608 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T13:41:57.113825Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:42:08.893485Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:42:12.475104Z info ads ADS: new connection for node:14 2026-11-03T13:42:12.476349Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T13:42:16.061424Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:43:00.187974Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T13:43:05.466Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 333 92 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "37a1ff5b-28a4-4877-9a18-f1d3fd3ed93d" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.132.0.112:8000" outbound|54321||e2e-prio-pool-c713431b-inference-pool-ip-634feed2.llm.svc.cluster.local 10.134.0.120:50048 10.134.0.120:443 10.132.0.19:54876 - v1-model-routing 2026-11-03T13:43:25.504674Z info ads ADS: "" 14 terminated with stream closed 2026-11-03T13:43:25.777182Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:43:30.502500Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=27 2026-11-03T13:43:30.761880Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T13:43:30.384Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 90 332 118 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "9866ebae-401d-4dea-9049-ea4c5032945a" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.132.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.120:45368 10.134.0.120:443 10.132.0.19:36652 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T13:43:48.388599Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:46:23.700608Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:46:50.838884Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:47:21.094077Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:47:35.330567Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:47:45.943616Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=27 [2026-11-03T13:47:45.827Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 84 326 116 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "84ce2027-cc1d-4297-9122-d6eab7512fdb" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.132.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.120:38938 10.134.0.120:443 10.132.0.19:35112 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T13:47:47.159925Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:47:59.880074Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:48:05.321380Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:48:25.541524Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:53:31.247229Z info ads ADS: new connection for node:15 2026-11-03T13:53:31.252718Z info cache read certificate from file resource=file-root:system 2026-11-03T13:53:35.127744Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:53:36.550766Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:53:40.731866Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"557","StreamId":"4116074280759164670"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=27 [2026-11-03T13:53:40.663Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 68 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "c1da2e32-dd07-4467-9cd6-dd638f263bc5" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.134.0.120:443 10.132.0.19:57354 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-11-03T13:53:40.994077Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-03T13:53:59.460187Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=27 2026-11-03T13:53:59.593870Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=27 2026-11-03T13:53:59.594119Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=27 [2026-11-03T13:53:59.398Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1724 196 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "d6124bef-1307-40b0-bebd-7843a728392e" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.120:44566 10.134.0.120:443 10.132.0.19:48154 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T13:54:22.089562Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-03T13:54:22.089854Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-11-03T13:54:21.888Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1720 201 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "7472bab8-84d7-433f-9cb8-e633aef509fb" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.120:36682 10.134.0.120:443 10.132.0.19:37012 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T13:54:24.436910Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=27 2026-11-03T13:54:24.437154Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=27 [2026-11-03T13:54:24.298Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1722 139 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "b1f83139-a984-4ac6-b5d1-6186b733c045" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.120:44566 10.134.0.120:443 10.132.0.19:33718 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T13:54:33.818222Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-03T13:54:33.818488Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-11-03T13:54:33.686Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1713 131 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "35603111-41b4-4e09-aebc-48bf71651da5" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.120:36682 10.134.0.120:443 10.132.0.19:57768 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T13:54:50.447103Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=26 2026-11-03T13:54:50.568478Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-03T13:54:50.568754Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-11-03T13:54:50.439Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1723 129 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "19e34e00-cc38-4da2-b60f-a3710495588a" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.120:36682 10.134.0.120:443 10.132.0.19:48062 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T13:54:52.898535Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-03T13:54:52.898779Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-11-03T13:54:52.773Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1721 125 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "3007877a-5136-49a5-9146-b27bd47a412b" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.120:36682 10.134.0.120:443 10.132.0.19:48084 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T13:55:08.332901Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:55:08.359439Z info ads ADS: "" 15 terminated with stream closed 2026-11-03T13:55:08.683315Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-03T13:55:08.524Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 74 316 159 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "a5827def-50e9-4567-9f9e-7261849f8564" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.132.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.120:46128 10.134.0.120:443 10.132.0.19:53868 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T13:55:11.361957Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:55:19.597060Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:55:41.335963Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:55:46.208325Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T13:55:50.263Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 305 78 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "9e18eba9-ae67-42f6-bba9-0d3d3ab328c3" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.134.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.120:58348 10.134.0.120:443 10.132.0.19:33534 - v1-completions-path [2026-11-03T13:56:05.532Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 289 81 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "5d1e1aa4-6636-49fc-8785-a9cb59e5331d" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.134.0.55:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.120:38938 10.134.0.120:443 10.132.0.19:53766 - v1-completions-path 2026-11-03T13:56:10.934473Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:56:25.557753Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T13:56:34.498Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 62 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "62ff295a-d408-4fe2-9b8d-5f14593f3426" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.134.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.120:58348 10.134.0.120:443 10.132.0.19:35898 - v1-completions-path [2026-11-03T13:56:37.536Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 59 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "3bef7797-7923-4ad0-ab04-bbf454284b48" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.134.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.120:58348 10.134.0.120:443 10.132.0.19:35914 - v1-completions-path [2026-11-03T13:56:38.785Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 56 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "6c3b8804-012b-4136-8309-52eef41a7f25" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.134.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.120:58348 10.134.0.120:443 10.132.0.19:35918 - v1-completions-path 2026-11-03T13:56:59.059205Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:57:10.495638Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:57:15.697050Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:57:44.605146Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:57:49.601103Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:57:53.541810Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-03T13:57:53.250Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 73 315 291 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "f29e50e8-ce70-4361-86c0-7686ca30f2e2" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.132.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.120:59744 10.134.0.120:443 10.132.0.19:51650 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T13:57:53.739Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 74 292 79 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "69767f08-d1a9-4a40-8459-bb98f36a14d2" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.132.0.43:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.134.0.120:57618 10.134.0.120:443 10.132.0.19:43396 - v1-completions-path 2026-11-03T13:58:04.043496Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:58:13.882094Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:58:25.235109Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:58:32.150622Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:58:47.707658Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:59:00.079779Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:59:04.035335Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-03T13:59:03.416Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 324 618 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "8ef2c9cc-65a7-4831-a7c7-f987d2388f49" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.132.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.120:60764 10.134.0.120:443 10.132.0.19:52506 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T13:59:24.531Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 76 0 33 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "ceb93d69-1979-41c9-b34d-82aa4d1723fb" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.120:443 10.132.0.19:48684 - v1-completions-path 2026-11-03T13:59:33.104364Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:59:56.276821Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:00:07.217478Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:02:47.625660Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-03T14:02:47.511Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 80 322 114 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "b5595ef1-cf53-401f-92b2-49eebbf16399" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.132.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.120:34678 10.134.0.120:443 10.132.0.19:47282 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T14:04:09.896018Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=27 [2026-11-03T14:04:09.778Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 313 117 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "2752faf0-2857-4059-8662-85cf4e94e3de" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.132.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.120:53628 10.134.0.120:443 10.132.0.19:52590 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T14:04:30.998415Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:04:49.704983Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:04:54.447485Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:05:08.182189Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:05:22.246292Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:05:31.971391Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:09:21.912906Z info cache event for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt : REMOVE, pushing to proxy 2026-11-03T14:12:45.402781Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:12:50.156423Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:14:37.626034Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:18:41.761870Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:18:47.143030Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T14:18:52.487Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 62 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "9e200fe0-e78b-4039-a8b0-c6388e880c9b" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.134.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.120:58348 10.134.0.120:443 10.132.0.19:55340 - v1-completions-path [2026-11-03T14:18:52.856Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 58 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "54511031-0efb-4bf1-b460-079f3da4b0ec" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.134.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.120:58348 10.134.0.120:443 10.132.0.19:55348 - v1-completions-path 2026-11-03T14:19:08.529612Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:19:19.295955Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:19:26.063280Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:19:37.854828Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-03T14:19:37.738Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 67 309 116 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "b0ce557f-15cd-4b90-9765-00ebb1c464ca" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.132.0.129:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.120:60246 10.134.0.120:443 10.132.0.19:37520 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T14:19:55.093981Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:20:01.671208Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:20:08.133286Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:20:16.033220Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-03T14:20:15.920Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 66 308 113 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "2c03f88a-aff2-410c-a069-98716bd44ac6" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.134.0.239:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.120:52126 10.134.0.120:443 10.132.0.19:52610 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T14:20:19.516Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 60 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "a316e47c-7d5a-458b-b255-7acb30458c8e" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.134.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.120:58348 10.134.0.120:443 10.132.0.19:52622 - v1-completions-path [2026-11-03T14:20:20.949Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 58 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "6931d92d-6095-47f5-a4c9-e2c4b469019e" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.134.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.120:58348 10.134.0.120:443 10.132.0.19:52636 - v1-completions-path [2026-11-03T14:20:23.102Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 60 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "8a5a1923-eef0-4b74-9ad1-585ca3a8f203" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.134.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.120:39408 10.134.0.120:443 10.132.0.19:45588 - v1-completions-path 2026-11-03T14:20:40.040639Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:20:50.736556Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:20:57.681992Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:21:03.845582Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:21:15.659975Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T14:21:22.466Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 64 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "0a00b33c-957b-44bb-a116-d4db5e8412ea" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.134.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.120:58348 10.134.0.120:443 10.132.0.19:53214 - v1-completions-path [2026-11-03T14:21:22.833Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 65 - "34.228.250.194,10.132.0.19" "python-requests/2.33.0" "ee5b040a-fef8-4aec-9033-907146c944dd" "maas.apps.135c31a7394c28a9259e.prod.konflux-ocp-ci.dev" "10.134.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.120:39408 10.134.0.120:443 10.132.0.19:53220 - v1-completions-path 2026-11-03T14:21:39.192571Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:21:42.954144Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=3 2026-11-03T14:21:50.035302Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:21:55.154620Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:22:13.344303Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:22:18.117490Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:22:40.388562Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:23:00.421913Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:23:11.483678Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:23:22.143685Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:23:33.207526Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:23:44.271375Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:23:55.344426Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:24:06.113768Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:24:16.964538Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:24:27.566824Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:24:38.374112Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8