2026-11-03T21:09:49.694698Z info FLAG: --concurrency="0" 2026-11-03T21:09:49.694719Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T21:09:49.694723Z info FLAG: --help="false" 2026-11-03T21:09:49.694725Z info FLAG: --log_as_json="false" 2026-11-03T21:09:49.694727Z info FLAG: --log_caller="" 2026-11-03T21:09:49.694729Z info FLAG: --log_output_level="default:info" 2026-11-03T21:09:49.694731Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T21:09:49.694735Z info FLAG: --log_target="[stdout]" 2026-11-03T21:09:49.694738Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T21:09:49.694739Z info FLAG: --outlierLogPath="" 2026-11-03T21:09:49.694742Z info FLAG: --profiling="true" 2026-11-03T21:09:49.694743Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T21:09:49.694746Z info FLAG: --proxyLogLevel="warning" 2026-11-03T21:09:49.694748Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T21:09:49.694750Z info FLAG: --stsPort="0" 2026-11-03T21:09:49.694752Z info FLAG: --templateFile="" 2026-11-03T21:09:49.694754Z info FLAG: --tokenManagerPlugin="" 2026-11-03T21:09:49.694757Z info FLAG: --vklog="0" 2026-11-03T21:09:49.694759Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T21:09:49.694909Z info Proxy role ips=[10.134.1.87] type=router id=maas-default-gateway-openshift-default-57f78bcd55-fr5rf.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T21:09:49.694971Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T21:09:49.696139Z info cpu limit detected as 2, setting concurrency 2026-11-03T21:09:49.696703Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T21:09:49.696715Z info JWT policy is third-party-jwt 2026-11-03T21:09:49.696718Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T21:09:49.698032Z info Opening status port 15020 2026-11-03T21:09:49.698044Z info Starting default Istio SDS Server 2026-11-03T21:09:49.698082Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T21:09:49.698108Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T21:09:49.699011Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T21:09:49.700936Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T21:09:49.702918Z info Starting proxy agent 2026-11-03T21:09:49.704079Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T21:09:49.703057Z info sds Starting SDS grpc server 2026-11-03T21:09:49.720955Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T21:09:49.775838Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T21:09:49.824078Z info cache generated new workload certificate resourceName=default latency=124.840764ms ttl=23h59m59.175926792s 2026-11-03T21:09:49.824114Z info cache Root cert has changed, start rotating root cert 2026-11-03T21:09:49.824149Z info cache returned workload trust anchor from cache ttl=23h59m59.175851562s 2026-11-03T21:09:49.955786Z info ads ADS: new connection for node:1 2026-11-03T21:09:49.955847Z info cache returned workload certificate from cache ttl=23h59m59.044153945s 2026-11-03T21:09:49.956242Z info ads ADS: new connection for node:2 2026-11-03T21:09:49.956317Z info cache returned workload trust anchor from cache ttl=23h59m59.043684425s 2026-11-03T21:09:50.042385Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:09:58.176055Z info Readiness succeeded in 8.48398267s 2026-11-03T21:09:58.176348Z info Envoy proxy is ready 2026-11-03T21:14:17.067068Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:14:17.067117Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:14:17.067135Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T21:14:17.067142Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T21:14:16.944Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 122 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "8cb7cdb2-7f4f-44bb-a0ab-e665350b0ec0" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.87:58414 10.134.1.87:443 10.132.0.11:56540 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:14:17.875018Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:14:44.074288Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:15:19.397018Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T21:15:25.440Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 140 405 90 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "2fb76877-7ba7-4b05-932d-842e2992b494" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.134.1.87:54536 10.134.1.87:443 10.132.0.11:40138 - v1-chat-completions-path 2026-11-03T21:15:39.279172Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:15:44.327168Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:15:53.452300Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:16:08.200687Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:16:16.144792Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:16:19.276022Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:16:19.276057Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:16:19.276069Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T21:16:19.276074Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T21:16:19.169Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 73 315 107 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "e257efb0-39c8-415f-991d-b3874f33a9b3" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.87:57356 10.134.1.87:443 10.132.0.11:60738 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:16:33.493496Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:16:45.237122Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:16:47.650842Z info ads ADS: new connection for node:3 2026-11-03T21:16:47.651079Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T21:16:47.651110Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T21:16:52.467032Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:17:31.369448Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T21:17:34.919Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 333 147 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "ab3dd5a5-900b-4ff8-b89d-c420234d3710" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.133.0.247:8000" outbound|54321||e2e-prio-pool-9f7f5f7b-inference-pool-ip-ca181a17.llm.svc.cluster.local 10.134.1.87:52602 10.134.1.87:443 10.132.0.11:53892 - v1-model-routing 2026-11-03T21:17:58.071330Z info ads ADS: "" 3 terminated 2026-11-03T21:17:58.343055Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:18:03.520740Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T21:18:03.061Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 287 76 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "cc270c48-b377-4c8e-98b9-3545e3827db5" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.133.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.1.87:44676 10.134.1.87:443 10.132.0.11:54668 - v1-completions-path 2026-11-03T21:18:17.305603Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:21:05.438300Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-11-03T21:21:05.454424Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:21:05.454458Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:21:05.454478Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T21:21:05.454485Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T21:21:05.334Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 33 326 119 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "f3f12435-24bf-449f-a3e4-56e9f30160bf" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.87:42444 10.134.1.87:443 10.132.0.11:45516 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:21:16.189278Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:21:42.046675Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:22:08.268767Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:22:23.023334Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:22:35.009984Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:22:47.858368Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:22:53.218917Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T21:22:57.653Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 72 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "cc7e04b8-47b4-41de-8d1e-069f277b4c95" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.87:46578 10.134.1.87:443 10.132.0.11:51352 - v1-completions-path [2026-11-03T21:22:59.834Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 64 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "bd03e651-4bc8-4eea-ace5-93f9ffe9816d" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.87:46578 10.134.1.87:443 10.132.0.11:51360 - v1-completions-path 2026-11-03T21:23:14.871135Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:24:03.673130Z info ads ADS: new connection for node:4 2026-11-03T21:24:03.677750Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-03T21:24:03.677811Z info cache read certificate from file resource=file-root:system 2026-11-03T21:24:07.765123Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:24:08.916914Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:24:40.747158Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:24:40.747195Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:24:40.747205Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T21:24:40.747210Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:24:40.747409Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T21:24:40.550Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1533 197 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "f0767c65-38bc-4596-96e2-e0e03dd2feb3" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.1.87:39254 10.134.1.87:443 10.132.0.11:45294 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T21:24:50.461876Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:24:50.461912Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:24:50.461926Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T21:24:50.461930Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:24:50.462121Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T21:24:50.265Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1537 196 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "38fc4dce-7ef0-4828-98a8-6eec1ebd12bb" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.1.87:59660 10.134.1.87:443 10.132.0.11:55814 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T21:24:55.125686Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:24:55.125717Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:24:55.125726Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T21:24:55.125731Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:24:55.125882Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T21:24:55.004Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1539 121 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "10d0ebf2-687c-4ef1-b425-e5b51e98110b" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.1.87:39254 10.134.1.87:443 10.132.0.11:55838 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T21:24:57.475356Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:24:57.475388Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:24:57.475398Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T21:24:57.475402Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:24:57.475556Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T21:24:57.345Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1539 130 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "b69a3db2-a947-41b5-8b10-a41397faa8c8" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.1.87:59660 10.134.1.87:443 10.132.0.11:49192 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T21:25:04.565148Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:25:04.565179Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:25:04.565189Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:25:04.565194Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T21:25:04.565341Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T21:25:04.442Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1538 123 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "37f71ea1-5645-46c5-89ed-0b8b4b45d83e" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.1.87:59660 10.134.1.87:443 10.132.0.11:49214 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T21:25:11.570516Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:25:11.570554Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:25:11.570568Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:25:11.570573Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T21:25:11.570742Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T21:25:11.444Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1539 126 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "379f1fe3-6bec-4dcd-8bd4-dfe848a0942f" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.1.87:59660 10.134.1.87:443 10.132.0.11:37382 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T21:25:50.557977Z info ads ADS: "" 4 terminated with stream closed 2026-11-03T21:25:50.602405Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:25:53.690305Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:26:04.874528Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:26:23.857907Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:26:28.819200Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T21:26:32.536Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 93 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "8f42080b-f895-457a-af6f-61b31aa8cb11" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.87:34750 10.134.1.87:443 10.132.0.11:56464 - v1-completions-path 2026-11-03T21:26:53.639917Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:27:06.718904Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:27:18.790823Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-11-03T21:27:18.735Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 62 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "9a4d3682-4deb-4be5-8c6f-5a6c1f488e92" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.87:34750 10.134.1.87:443 10.132.0.11:51536 - v1-completions-path [2026-11-03T21:27:19.836Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 61 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "8b24bd75-3288-4a5b-a065-051f50c7a261" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.87:34750 10.134.1.87:443 10.132.0.11:51558 - v1-completions-path 2026-11-03T21:27:37.817756Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:27:49.099904Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:27:54.113535Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:28:23.585655Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:28:28.406020Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:28:40.992515Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:28:51.522198Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:29:02.739395Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:29:09.662617Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:29:24.984380Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:29:38.273268Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:29:44.993108Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:30:03.702274Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:30:09.583339Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:30:21.506523Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:30:47.080320Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:30:58.173537Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:31:18.541400Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:31:26.935235Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:33:52.051446Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:33:57.291765Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:35:05.874343Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T21:35:05.874372Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T21:35:05.874376Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T21:35:05.874379Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-11-03T21:35:05.823Z] "POST /llm/e2e-trlp-test-simulated/v1/completions HTTP/1.1" 403 - - "-" 73 113 50 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "eb6f0873-36b4-4322-a115-b07b8c685feb" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-trlp-test-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.87:443 10.132.0.11:34718 - v1-completions-path 2026-11-03T21:35:27.385768Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:35:45.158997Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:35:50.149671Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:36:02.779517Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:36:19.607036Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:36:23.465540Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:36:29.432860Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:36:29.432898Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:36:29.432912Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:36:29.432917Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T21:36:29.326Z] "DELETE /v1/api-keys/0a563dbc-7235-4d6e-badf-2dbb16c2608a HTTP/1.1" 200 - via_upstream - "-" 0 369 106 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "8b37a900-7bc1-4308-89c3-01e17340a39e" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.1.110:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.87:33356 10.134.1.87:443 10.132.0.11:54002 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:36:43.234779Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:36:53.946509Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:37:00.845272Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T21:37:13.373Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 72 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "f5517db8-bbab-4642-9f61-60b4bfb5d756" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.87:46578 10.134.1.87:443 10.132.0.11:43830 - v1-completions-path 2026-11-03T21:37:29.636396Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:37:36.439558Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:37:43.091737Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:38:11.548732Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:38:22.190639Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:38:29.156329Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:38:35.942162Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T21:38:43.199Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 78 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "e9273d51-ed82-458f-b7bd-c51be276d77c" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.87:46578 10.134.1.87:443 10.132.0.11:37444 - v1-completions-path 2026-11-03T21:38:46.988125Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:39:10.364459Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:39:23.501763Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:39:33.852391Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:39:40.689345Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:40:03.571978Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:40:25.570507Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:40:47.728104Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:41:07.327856Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:41:18.075653Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:41:28.697763Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:41:39.636745Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:41:50.442156Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:42:00.991642Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T21:42:18.023477Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2