2026-11-02T14:18:24.089369Z info FLAG: --concurrency="0" 2026-11-02T14:18:24.089655Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-02T14:18:24.089704Z info FLAG: --help="false" 2026-11-02T14:18:24.089729Z info FLAG: --log_as_json="false" 2026-11-02T14:18:24.089751Z info FLAG: --log_caller="" 2026-11-02T14:18:24.089772Z info FLAG: --log_output_level="default:info" 2026-11-02T14:18:24.090012Z info FLAG: --log_stacktrace_level="default:none" 2026-11-02T14:18:24.090039Z info FLAG: --log_target="[stdout]" 2026-11-02T14:18:24.090044Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-02T14:18:24.090049Z info FLAG: --outlierLogPath="" 2026-11-02T14:18:24.090060Z info FLAG: --profiling="true" 2026-11-02T14:18:24.090071Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-02T14:18:24.090076Z info FLAG: --proxyLogLevel="warning" 2026-11-02T14:18:24.090089Z info FLAG: --serviceCluster="istio-proxy" 2026-11-02T14:18:24.090097Z info FLAG: --stsPort="0" 2026-11-02T14:18:24.090107Z info FLAG: --templateFile="" 2026-11-02T14:18:24.090111Z info FLAG: --tokenManagerPlugin="" 2026-11-02T14:18:24.090120Z info FLAG: --vklog="0" 2026-11-02T14:18:24.090125Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-02T14:18:24.090339Z info Proxy role ips=[10.134.0.118] type=router id=maas-default-gateway-openshift-default-55fbd65575-zdbqj.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-02T14:18:24.090481Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-02T14:18:24.093539Z info cpu limit detected as 2, setting concurrency 2026-11-02T14:18:24.093910Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-02T14:18:24.093991Z info JWT policy is third-party-jwt 2026-11-02T14:18:24.094017Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-02T14:18:24.095868Z info Starting default Istio SDS Server 2026-11-02T14:18:24.095935Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-02T14:18:24.095980Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-02T14:18:24.096184Z info Opening status port 15020 2026-11-02T14:18:24.097085Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-02T14:18:24.100246Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-02T14:18:24.104073Z info Starting proxy agent 2026-11-02T14:18:24.104178Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-02T14:18:24.118415Z info sds Starting SDS grpc server 2026-11-02T14:18:24.118447Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-02T14:18:24.252672Z info cache generated new workload certificate resourceName=default latency=150.0844ms ttl=23h59m59.747333286s 2026-11-02T14:18:24.252792Z info cache Root cert has changed, start rotating root cert 2026-11-02T14:18:24.252881Z info cache returned workload trust anchor from cache ttl=23h59m59.747119685s 2026-11-02T14:18:24.282469Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-02T14:18:24.559985Z info ads ADS: new connection for node:1 2026-11-02T14:18:24.566268Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-02T14:18:24.566316Z info cache read certificate from file resource=file-root:system 2026-11-02T14:18:24.594823Z info ads ADS: new connection for node:2 2026-11-02T14:18:24.594904Z info cache returned workload certificate from cache ttl=23h59m59.405098834s 2026-11-02T14:18:24.595688Z info ads ADS: new connection for node:3 2026-11-02T14:18:24.595765Z info cache returned workload trust anchor from cache ttl=23h59m59.404236106s 2026-11-02T14:18:24.716222Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:18:33.897302Z info Readiness succeeded in 9.812463282s 2026-11-02T14:18:33.897787Z info Envoy proxy is ready 2026-11-02T14:19:07.296986Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:19:22.504304Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:21:13.918913Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:21:38.578303Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T14:21:44.247Z] "POST /llm/e2e-dft-c484d7/anything/e2e-dft-c484d7-57154c HTTP/1.1" 401 - - "-" 109 88 86 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "1e5e042a-947b-4ee8-ad9c-0cc4d46e1d3f" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-c484d7.llm.svc.cluster.local - 10.134.0.118:443 10.132.0.13:42632 - llm.external-model-e2e-dft-c484d7-41f2418f9e17.2 2026-11-02T14:21:56.947943Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:22:09.848721Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:22:11.397220Z info ads ADS: new connection for node:4 2026-11-02T14:22:11.397437Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-02T14:22:11.397463Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-02T14:22:14.200997Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:22:18.198735Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:22:23.831473Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:22:28.322656Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:22:31.331473Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:22:31.807182Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:22:34.066797Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:22:58.565031Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:23:05.521099Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:23:20.556727Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:23:40.677323Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:23:45.319454Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T14:23:45.319631Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T14:23:45.319681Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T14:23:45.319711Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T14:23:45.320115Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-02T14:23:44.925Z] "POST /anything/e2e-rt-sw-0ec724-client HTTP/1.1" 200 - via_upstream - "-" 124 1769 395 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "c77c247f-76b3-47cd-bbfb-5914a31df777" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.118:50480 10.134.0.118:443 10.132.0.13:45978 - llm.external-model-e2e-rt-sw-0ec724-bc1398cbe93f.2 2026-11-02T14:23:56.560443Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:24:06.621908Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T14:24:06.621972Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T14:24:06.621984Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T14:24:06.621989Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T14:24:06.622188Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-02T14:24:06.495Z] "POST /anything/e2e-rt-sw-0ec724-client HTTP/1.1" 200 - via_upstream - "-" 124 1766 126 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "dffc6927-f5d7-4461-a5f9-ba63e4f354da" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.118:35956 10.134.0.118:443 10.132.0.13:34318 - llm.external-model-e2e-rt-sw-0ec724-bc1398cbe93f.0 2026-11-02T14:24:16.757478Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:25:00.063400Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T14:25:01.045Z] "POST /anything/e2e-rt-eq-0c8e39-client HTTP/1.1" 200 - via_upstream - "-" 124 1513 70 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "df9c03e7-89a0-4ba2-8fe6-e655a42efa17" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.118:34736 10.134.0.118:443 10.132.0.13:57572 - llm.external-model-e2e-rt-eq-0c8e39-8f80e74632e1.0 2026-11-02T14:25:03.452099Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T14:25:03.452146Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T14:25:03.452162Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T14:25:03.452168Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T14:25:03.452532Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 2026-11-02T14:25:04.094929Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T14:25:04.094977Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T14:25:04.094989Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T14:25:04.094993Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T14:25:04.095194Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-02T14:25:03.322Z] "POST /anything/e2e-rt-eq-0c8e39-client HTTP/1.1" 200 - via_upstream - "-" 124 1760 129 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "f47310d1-ae49-46b7-bea3-4be8f15e3e60" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.118:50896 10.134.0.118:443 10.132.0.13:57580 - llm.external-model-e2e-rt-eq-0c8e39-8f80e74632e1.0 [2026-11-02T14:25:03.990Z] "POST /anything/e2e-rt-eq-0c8e39-client HTTP/1.1" 200 - via_upstream - "-" 124 1761 104 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "a7ca25ec-3303-4047-b510-6498b148d5e6" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.118:50906 10.134.0.118:443 10.132.0.13:57592 - llm.external-model-e2e-rt-eq-0c8e39-8f80e74632e1.2 2026-11-02T14:25:04.393868Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T14:25:04.393900Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T14:25:04.393912Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T14:25:04.393925Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T14:25:04.394155Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-02T14:25:04.296Z] "POST /anything/e2e-rt-eq-0c8e39-client HTTP/1.1" 200 - via_upstream - "-" 124 1765 97 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "65f0abea-5719-4de3-bbcf-154c683bd080" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.118:50896 10.134.0.118:443 10.132.0.13:41362 - llm.external-model-e2e-rt-eq-0c8e39-8f80e74632e1.2 2026-11-02T14:25:28.512398Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:25:33.697154Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:25:48.848696Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:27:57.247447Z info ads ADS: "" 4 terminated with stream closed 2026-11-02T14:28:56.726745Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:29:16.555294Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:29:21.431240Z info ads ADS: "" 1 terminated with stream closed 2026-11-02T14:29:37.606345Z info ads ADS: new connection for node:5 2026-11-02T14:29:37.663649Z info cache read certificate from file resource=file-root:system 2026-11-02T14:30:58.908272Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:31:00.927117Z info ads ADS: "" 5 terminated with stream closed 2026-11-02T14:32:35.828716Z info ads ADS: new connection for node:6 2026-11-02T14:32:35.835043Z info cache read certificate from file resource=file-root:system 2026-11-02T14:35:25.459389Z info ads ADS: "" 6 terminated with stream closed 2026-11-02T14:36:56.167530Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:37:32.008471Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:37:50.050825Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-02T14:38:51.747596Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:43:41.872956Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:44:08.100284Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:46:20.026008Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:46:43.437450Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:46:45.722489Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:46:58.225703Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:47:02.435529Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:47:19.739966Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:47:40.160480Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:47:50.751107Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:48:01.061000Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:48:11.531800Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:48:21.934745Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:48:32.339300Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:48:42.731424Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:48:53.158585Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:49:03.603803Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:49:14.017771Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:49:24.264600Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:50:07.716536Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:50:10.730232Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T14:50:10.730323Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T14:50:10.730347Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T14:50:10.730373Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T14:50:10.967741Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-02T14:50:10.642Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 61 303 88 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "99d25b45-e3db-40e4-a528-25b9480dd0f8" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "10.132.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.118:33920 10.134.0.118:443 10.132.0.13:34992 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T14:50:10.885Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 2817 3089 83 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "1e4679cd-8e31-4661-aa17-1cf1c4ae77aa" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "10.132.0.42:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.134.0.118:56426 10.134.0.118:443 10.132.0.13:34994 - v1-chat-completions-path 2026-11-02T14:50:22.963571Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-02T14:50:22.963600Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-02T14:50:22.963604Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-02T14:50:22.963609Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 2026-11-02T14:50:23.563628Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T14:50:22.888Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 403 - - "-" 61 103 75 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "ebd2df01-ca16-471a-8d83-10819eb6902e" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.118:443 10.132.0.13:53676 - v1-catch-all-path 2026-11-02T14:50:26.004932Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:50:32.675875Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:50:35.870929Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T14:50:35.870979Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T14:50:35.871018Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T14:50:35.871025Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-02T14:50:35.792Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 317 78 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "c66f3f15-9e4a-4b05-a668-83625884ea7a" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "10.132.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.118:33920 10.134.0.118:443 10.132.0.13:48202 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T14:50:39.520717Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:50:49.948420Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:51:04.993311Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:51:15.944133Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:51:18.022650Z info ads ADS: new connection for node:7 2026-11-02T14:51:18.022824Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-02T14:51:20.077999Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:52:00.633421Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T14:52:04.774Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 333 91 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "de697c62-9523-44cf-a0d1-7e9b5ff56144" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "10.132.0.151:8000" outbound|54321||e2e-prio-pool-170c8925-inference-pool-ip-0b8de156.llm.svc.cluster.local 10.134.0.118:40330 10.134.0.118:443 10.132.0.13:58650 - v1-model-routing [2026-11-02T14:52:05.455Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 344 63 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "af884554-a849-4cbe-84af-07dd68be277c" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "10.132.0.151:8000" outbound|54321||e2e-prio-pool-170c8925-inference-pool-ip-0b8de156.llm.svc.cluster.local 10.134.0.118:40330 10.134.0.118:443 10.132.0.13:58664 - v1-model-routing 2026-11-02T14:52:20.147944Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:52:20.220687Z info ads ADS: "" 7 terminated 2026-11-02T14:52:22.680710Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:52:25.742693Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:52:31.460389Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:52:36.884492Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-02T14:52:36.884574Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T14:52:36.884597Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T14:52:36.884617Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T14:52:36.884624Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T14:52:36.806Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 78 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "9365e64a-bf85-4a93-80e9-df01453cc07d" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "10.132.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.118:37508 10.134.0.118:443 10.132.0.13:39492 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T14:52:43.148900Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:53:00.126551Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:53:25.853659Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:53:40.270716Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:53:44.615611Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:53:56.454030Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:54:01.077058Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:54:04.126973Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T14:54:04.127011Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T14:54:04.127028Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T14:54:04.127033Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-02T14:54:04.071Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 524 55 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "4e0a21d5-9a3b-4846-b598-0c4c33424caa" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "10.132.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.118:36434 10.134.0.118:443 10.132.0.13:33876 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T14:54:04.270Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 74 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "5cce989f-c674-4ee4-8ba0-909c655e8704" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "10.134.0.42:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.118:59596 10.134.0.118:443 10.132.0.13:60100 - v1-completions-path 2026-11-02T14:54:16.836278Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T14:58:13.693Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 290 92 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "fc004144-9f10-4229-a864-0ac574a76eef" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "10.134.0.43:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.118:40004 10.134.0.118:443 10.132.0.13:60330 - v1-completions-path 2026-11-02T14:58:57.487386Z info ads ADS: new connection for node:8 2026-11-02T14:58:57.492751Z info cache read certificate from file resource=file-root:system 2026-11-02T14:58:58.643522Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:59:00.289677Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T14:59:22.285817Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T14:59:22.285872Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T14:59:22.285889Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T14:59:22.285895Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T14:59:22.286126Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-02T14:59:22.164Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1711 121 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "fb964674-9b09-4cc7-9eac-880eee34c38c" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.118:52900 10.134.0.118:443 10.132.0.13:41064 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T14:59:24.461195Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T14:59:24.461234Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T14:59:24.461248Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T14:59:24.461254Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T14:59:24.461480Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-02T14:59:24.363Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1718 98 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "e1ec15b6-d1c0-4a61-9b22-d25ccb1460be" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.118:52900 10.134.0.118:443 10.132.0.13:52280 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T14:59:48.042101Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T14:59:48.042131Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T14:59:48.042142Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T14:59:48.042147Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T14:59:48.042313Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-02T14:59:47.943Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1718 98 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "57675611-6fb7-9367-a013-72c92eb7fd12" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.118:52900 10.134.0.118:443 10.132.0.13:41268 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T14:59:50.174444Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T14:59:50.174482Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T14:59:50.174494Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T14:59:50.174498Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T14:59:50.174702Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-02T14:59:50.067Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1718 107 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "8a58081b-8d56-4137-be71-a3b2c19ef58f" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.118:58702 10.134.0.118:443 10.132.0.13:41290 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T15:00:16.581147Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T15:00:16.581186Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T15:00:16.581199Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T15:00:16.581205Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T15:00:16.581459Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-02T15:00:16.482Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1721 98 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "ec9ba850-5633-487e-bc22-97a38f5aac60" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.118:52900 10.134.0.118:443 10.132.0.13:52014 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T15:00:46.595778Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:00:46.748347Z info ads ADS: "" 8 terminated 2026-11-02T15:00:48.186412Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:00:55.002650Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:01:10.767917Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:01:26.941526Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T15:01:32.100Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 288 66 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "41bedb1c-db2c-4ad3-b082-4b528dcbe734" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "10.134.0.42:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.118:36640 10.134.0.118:443 10.132.0.13:34834 - v1-completions-path [2026-11-02T15:01:33.265Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 62 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "5d337a7d-1ac6-423a-8d85-e48c85098c66" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "10.134.0.42:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.118:59596 10.134.0.118:443 10.132.0.13:34840 - v1-completions-path [2026-11-02T15:01:33.839Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 60 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "a7ce424d-cc8c-4a4d-aa4a-f7b04ff5e9c2" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "10.134.0.42:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.118:36640 10.134.0.118:443 10.132.0.13:34844 - v1-completions-path [2026-11-02T15:01:34.001Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 57 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "1e02c0b6-06a3-4801-97e9-6fc18473dc65" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "10.134.0.42:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.118:36640 10.134.0.118:443 10.132.0.13:34852 - v1-completions-path 2026-11-02T15:01:50.479082Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:02:00.785065Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:02:05.553287Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T15:02:09.568Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 62 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "ccdd3fe1-472a-426d-9a80-05572a588bbe" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "10.134.0.42:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.118:59596 10.134.0.118:443 10.132.0.13:60138 - v1-completions-path [2026-11-02T15:02:09.972Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 59 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "fbbcc59f-b94e-4a27-8add-1eef672bbaf3" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "10.134.0.42:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.118:59596 10.134.0.118:443 10.132.0.13:60140 - v1-completions-path 2026-11-02T15:02:22.631043Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:02:24.991339Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:02:28.131515Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T15:02:28.131664Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T15:02:28.131726Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T15:02:28.131760Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-02T15:02:25.768Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 73 316 2363 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "a3529649-535b-4443-a35f-5626120b8569" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "10.132.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.118:59530 10.134.0.118:443 10.132.0.13:58030 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T15:02:32.641857Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:02:44.756227Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:02:50.290780Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:02:58.327082Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:03:04.013155Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:03:22.003598Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:03:25.247032Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:03:54.397946Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T15:03:54.397982Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T15:03:54.397995Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T15:03:54.398001Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T15:03:54.498032Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T15:03:54.498074Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T15:03:54.498093Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T15:03:54.498098Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-02T15:03:54.321Z] "DELETE /v1/api-keys/ff3ec684-e441-42ba-8186-2032c74726c2 HTTP/1.1" 404 - via_upstream - "-" 0 29 76 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "807f76ed-0572-4944-9cea-460c7552b5b0" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "10.132.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.118:56446 10.134.0.118:443 10.132.0.13:35556 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T15:03:54.416Z] "DELETE /v1/api-keys/b63f9199-fbd7-421a-8b15-cbd7a59dc03c HTTP/1.1" 200 - via_upstream - "-" 0 375 81 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "5e7db7c8-7d28-456e-8c86-d2a92d557f53" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "10.132.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.118:56446 10.134.0.118:443 10.132.0.13:35568 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T15:04:01.415412Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:04:19.641983Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:04:29.894087Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T15:04:44.806Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 37 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "2d79f8ff-83ec-41a2-92e9-030b9ef2755b" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.118:443 10.132.0.13:52204 - v1-completions-path 2026-11-02T15:06:22.447862Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=3 2026-11-02T15:06:30.728291Z info cache event for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt : REMOVE, pushing to proxy 2026-11-02T15:07:00.187744Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T15:07:00.187781Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T15:07:00.187800Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T15:07:00.187805Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T15:07:00.107Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 80 322 79 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "af469849-ad89-4766-8ae3-a0cfc0506cbb" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "10.132.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.118:40702 10.134.0.118:443 10.132.0.13:60832 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T15:07:17.843596Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:07:32.859089Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:07:45.318788Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:07:48.715909Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:08:04.952051Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:08:15.335671Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:08:30.161959Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T15:08:34.255Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 59 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "4b588182-94d7-44a8-9dd3-6fb3c6c16abc" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "10.134.0.42:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.118:59596 10.134.0.118:443 10.132.0.13:49388 - v1-completions-path 2026-11-02T15:08:35.408682Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T15:08:35.408720Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T15:08:35.408736Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T15:08:35.408741Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T15:08:35.332Z] "DELETE /v1/api-keys/85627542-d762-4aef-9d28-7a0c012075a4 HTTP/1.1" 200 - via_upstream - "-" 0 365 76 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "1c0cf60a-3ec5-4736-8300-a51f0d954ad6" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "10.132.0.174:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.118:41444 10.134.0.118:443 10.132.0.13:43492 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T15:08:47.484527Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:08:52.145906Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:08:55.545070Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T15:09:06.053Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 59 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "a5f82f1e-6d50-46ce-a7a0-747e9b691ca9" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.118:443 10.132.0.13:48382 - v1-completions-path 2026-11-02T15:09:18.305832Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:09:28.655017Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:09:33.277586Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:09:45.710947Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:09:50.216037Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:10:08.573155Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T15:10:19.057962Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T15:10:36.005Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 73 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "8ed5cc57-9c0f-40a2-900c-2eefcd7bdd51" "maas.apps.ca30a64395dcfc3deb10.prod.konflux-ocp-ci.dev" "10.134.0.43:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.118:40004 10.134.0.118:443 10.132.0.13:42498 - v1-completions-path