2026-10-30T04:49:30.513527Z info FLAG: --concurrency="0" 2026-10-30T04:49:30.513545Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T04:49:30.513549Z info FLAG: --help="false" 2026-10-30T04:49:30.513551Z info FLAG: --log_as_json="false" 2026-10-30T04:49:30.513554Z info FLAG: --log_caller="" 2026-10-30T04:49:30.513555Z info FLAG: --log_output_level="default:info" 2026-10-30T04:49:30.513557Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T04:49:30.513562Z info FLAG: --log_target="[stdout]" 2026-10-30T04:49:30.513565Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T04:49:30.513567Z info FLAG: --outlierLogPath="" 2026-10-30T04:49:30.513569Z info FLAG: --profiling="true" 2026-10-30T04:49:30.513571Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T04:49:30.513573Z info FLAG: --proxyLogLevel="warning" 2026-10-30T04:49:30.513575Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T04:49:30.513577Z info FLAG: --stsPort="0" 2026-10-30T04:49:30.513579Z info FLAG: --templateFile="" 2026-10-30T04:49:30.513581Z info FLAG: --tokenManagerPlugin="" 2026-10-30T04:49:30.513583Z info FLAG: --vklog="0" 2026-10-30T04:49:30.513585Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T04:49:30.513733Z info Proxy role ips=[10.132.1.30] type=router id=maas-default-gateway-openshift-default-746ff8f49f-mst95.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T04:49:30.513780Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T04:49:30.514812Z info cpu limit detected as 2, setting concurrency 2026-10-30T04:49:30.515004Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T04:49:30.515014Z info JWT policy is third-party-jwt 2026-10-30T04:49:30.515017Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T04:49:30.516853Z info Starting default Istio SDS Server 2026-10-30T04:49:30.516872Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T04:49:30.516889Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T04:49:30.517730Z info Opening status port 15020 2026-10-30T04:49:30.518263Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T04:49:30.519713Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T04:49:30.520594Z info Starting proxy agent 2026-10-30T04:49:30.520640Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T04:49:30.529735Z info sds Starting SDS grpc server 2026-10-30T04:49:30.529756Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T04:49:30.584737Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T04:49:30.602905Z info cache generated new workload certificate resourceName=default latency=84.944265ms ttl=23h59m59.397097527s 2026-10-30T04:49:30.602935Z info cache Root cert has changed, start rotating root cert 2026-10-30T04:49:30.602969Z info cache returned workload trust anchor from cache ttl=23h59m59.397031324s 2026-10-30T04:49:30.802570Z info ads ADS: new connection for node:1 2026-10-30T04:49:30.808804Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-10-30T04:49:30.808856Z info cache read certificate from file resource=file-root:system 2026-10-30T04:49:30.843860Z info ads ADS: new connection for node:2 2026-10-30T04:49:30.843923Z info cache returned workload certificate from cache ttl=23h59m59.156079208s 2026-10-30T04:49:30.845010Z info ads ADS: new connection for node:3 2026-10-30T04:49:30.845080Z info cache returned workload trust anchor from cache ttl=23h59m59.154921454s 2026-10-30T04:49:30.958783Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:49:39.152117Z info Readiness succeeded in 8.640831147s 2026-10-30T04:49:39.152598Z info Envoy proxy is ready 2026-10-30T04:49:39.226352Z info ads ADS: new connection for node:4 2026-10-30T04:49:39.226580Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T04:49:39.226616Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T04:52:46.060419Z info ads ADS: "" 4 terminated with stream closed 2026-10-30T04:54:24.417178Z info ads ADS: "" 1 terminated with stream closed 2026-10-30T05:00:26.417201Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:01:13.255913Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:04:32.178675Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:06:03.286779Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:14:37.407182Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T05:14:37.407234Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T05:14:37.407249Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T05:14:37.407254Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T05:14:37.309Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 30 314 97 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "2b620c92-82af-491e-9d48-7c7e3f80d38f" "maas.apps.59fbafd611cf6c800ee5.prod.konflux-ocp-ci.dev" "10.134.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.30:41714 10.132.1.30:443 10.132.0.17:55880 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T05:14:40.809187Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:15:05.909893Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:15:39.247811Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T05:15:44.731Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 147 412 91 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "fb817b1c-db05-4e29-a3bb-b3aa26bb60a8" "maas.apps.59fbafd611cf6c800ee5.prod.konflux-ocp-ci.dev" "10.134.0.43:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.132.1.30:38704 10.132.1.30:443 10.132.0.17:39270 - v1-chat-completions-path 2026-10-30T05:15:57.549398Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:15:59.813554Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:16:07.661088Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T05:16:07.661125Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T05:16:07.661139Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T05:16:07.661144Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T05:16:07.586Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 323 74 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "44e0b4ad-6f0e-4062-8638-d7842373d9ad" "maas.apps.59fbafd611cf6c800ee5.prod.konflux-ocp-ci.dev" "10.134.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.30:51926 10.132.1.30:443 10.132.0.17:40428 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T05:16:10.460974Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:16:15.760712Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T05:16:15.760749Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T05:16:15.760762Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T05:16:15.760767Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T05:16:15.690Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 317 70 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "9aaddbb9-1625-4ee6-9dc1-c395de326bdb" "maas.apps.59fbafd611cf6c800ee5.prod.konflux-ocp-ci.dev" "10.134.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.30:48476 10.132.1.30:443 10.132.0.17:37774 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T05:16:16.055502Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-10-30T05:16:15.975Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 80 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "83f38a34-bf2a-4ba8-8e25-d4190d18aec6" "maas.apps.59fbafd611cf6c800ee5.prod.konflux-ocp-ci.dev" "10.134.0.42:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.1.30:59506 10.132.1.30:443 10.132.0.17:37790 - v1-catch-all-path 2026-10-30T05:16:28.523895Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:16:30.754998Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:16:45.615079Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:16:56.556755Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:16:58.853771Z info ads ADS: new connection for node:5 2026-10-30T05:16:58.854025Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T05:17:01.893661Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:17:40.644918Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:17:43.493414Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T05:17:43.493460Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T05:17:43.493481Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T05:17:43.493488Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T05:17:41.417Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 2075 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "787edad2-2fdb-4f2c-b1f5-6facd385f198" "maas.apps.59fbafd611cf6c800ee5.prod.konflux-ocp-ci.dev" "10.134.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.30:57506 10.132.1.30:443 10.132.0.17:39392 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T05:17:59.708070Z info ads ADS: "" 5 terminated 2026-10-30T05:17:59.761293Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:18:01.975652Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T05:18:01.975688Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T05:18:01.975702Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T05:18:01.975707Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T05:18:01.901Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 90 333 74 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "b799e7a0-82cb-4cef-8de3-ea3eee9cddbc" "maas.apps.59fbafd611cf6c800ee5.prod.konflux-ocp-ci.dev" "10.134.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.30:38664 10.132.1.30:443 10.132.0.17:34358 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T05:18:03.122618Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:18:10.631469Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:19:43.299777Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:20:00.768247Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:20:15.675752Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:20:27.744495Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:20:31.545723Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:20:39.461984Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-10-30T05:20:42.851158Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:20:47.264058Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:21:05.415564Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:21:52.108483Z info ads ADS: new connection for node:6 2026-10-30T05:21:52.113912Z info cache read certificate from file resource=file-root:system 2026-10-30T05:21:53.197853Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:21:54.677423Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:21:57.097268Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-10-30T05:21:57.140016Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T05:21:57.140160Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T05:21:57.140220Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T05:21:57.140251Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T05:21:55.477Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 73 316 1663 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "09820257-4ed2-412f-acf3-ffd7a3c6cefb" "maas.apps.59fbafd611cf6c800ee5.prod.konflux-ocp-ci.dev" "10.134.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.30:53616 10.132.1.30:443 10.132.0.17:57478 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T05:22:59.288702Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T05:22:59.288737Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T05:22:59.288748Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T05:22:59.288753Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T05:22:59.288906Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-10-30T05:22:59.181Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1727 107 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "ad1c1ff1-4de0-4c6b-b2f7-aa6749325850" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.1.30:49438 10.132.1.30:443 10.132.0.17:58296 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T05:23:14.147905Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T05:23:14.147942Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T05:23:14.147953Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T05:23:14.147959Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T05:23:14.148139Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T05:23:14.037Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1729 110 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "5b2ca627-ea09-4b4d-b53d-85cbb188d019" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.1.30:43416 10.132.1.30:443 10.132.0.17:47350 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T05:23:29.618019Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:23:29.729968Z info ads ADS: "" 6 terminated 2026-10-30T05:23:31.091794Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:23:37.944905Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:23:54.937123Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:23:59.350265Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:24:03.093680Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-10-30T05:24:03.087Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 54 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "54b4ef88-1c4f-4544-aab4-56e25ea7698c" "maas.apps.59fbafd611cf6c800ee5.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.1.30:443 10.132.0.17:41222 - v1-completions-path 2026-10-30T05:24:15.130937Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:24:30.911404Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T05:24:37.684Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 72 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "ffe91318-895a-4975-a2a5-2b03ef0b4c99" "maas.apps.59fbafd611cf6c800ee5.prod.konflux-ocp-ci.dev" "10.132.0.46:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.30:45344 10.132.1.30:443 10.132.0.17:53422 - v1-completions-path 2026-10-30T05:24:54.044612Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:25:04.318143Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:25:08.804132Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:25:25.859515Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:25:28.181977Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:25:47.292305Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:25:51.802443Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:26:07.466593Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:26:09.805609Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:26:27.472370Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:26:38.352253Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:26:41.336341Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:27:00.537992Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:27:03.470596Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:27:08.578579Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T05:27:08.578618Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T05:27:08.578634Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T05:27:08.578639Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T05:27:08.502Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 89 331 75 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "9712bb26-9d9a-41a4-b643-b5a382c53927" "maas.apps.59fbafd611cf6c800ee5.prod.konflux-ocp-ci.dev" "10.134.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.30:56950 10.132.1.30:443 10.132.0.17:55272 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T05:27:10.206Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 76 0 32 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "9f97d4d6-ca93-4e67-a3f5-27b03677d896" "maas.apps.59fbafd611cf6c800ee5.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.1.30:443 10.132.0.17:55276 - v1-completions-path 2026-10-30T05:27:15.559055Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:27:33.475830Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:27:43.778784Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:28:03.345457Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:28:06.643225Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:29:51.051021Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:29:54.449919Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:31:29.101226Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T05:31:31.375Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 73 289 438 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "ee414ba3-e496-4d45-b99c-3b354c77369c" "maas.apps.59fbafd611cf6c800ee5.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-trlp-test-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.1.30:39122 10.132.1.30:443 10.132.0.17:34528 - v1-completions-path 2026-10-30T05:31:44.518908Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:31:47.045546Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:32:05.842884Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:32:17.609351Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:32:21.122888Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T05:32:24.264Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 64 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "1588ea19-6b93-48bc-8c18-15a4a9d753c5" "maas.apps.59fbafd611cf6c800ee5.prod.konflux-ocp-ci.dev" "10.132.0.46:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.30:41772 10.132.1.30:443 10.132.0.17:59036 - v1-completions-path 2026-10-30T05:32:37.004329Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:32:47.481708Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:32:52.515033Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T05:32:56.771Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 67 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "b3e4ce27-bf03-4062-a003-e26050e78788" "maas.apps.59fbafd611cf6c800ee5.prod.konflux-ocp-ci.dev" "10.132.0.46:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.30:41772 10.132.1.30:443 10.132.0.17:40582 - v1-completions-path 2026-10-30T05:33:09.416627Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:33:13.812516Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:33:17.422502Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T05:33:25.105Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 63 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "925c7455-864e-4359-bf4c-c315bb2e327e" "maas.apps.59fbafd611cf6c800ee5.prod.konflux-ocp-ci.dev" "10.132.0.46:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.30:41772 10.132.1.30:443 10.132.0.17:42586 - v1-completions-path [2026-10-30T05:33:25.471Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 61 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "223b7f5c-4612-48ac-95ad-d053173bf805" "maas.apps.59fbafd611cf6c800ee5.prod.konflux-ocp-ci.dev" "10.132.0.46:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.30:45344 10.132.1.30:443 10.132.0.17:42590 - v1-completions-path [2026-10-30T05:33:26.381Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 62 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "3fcd064e-d213-494b-80e3-c336a517713c" "maas.apps.59fbafd611cf6c800ee5.prod.konflux-ocp-ci.dev" "10.132.0.46:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.30:41772 10.132.1.30:443 10.132.0.17:42606 - v1-completions-path 2026-10-30T05:33:40.047768Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:33:50.443695Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:33:54.899337Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:33:57.891602Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:34:00.879488Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T05:34:05.367Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 69 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "1bff210e-c449-49de-bf88-c23d9bd350f6" "maas.apps.59fbafd611cf6c800ee5.prod.konflux-ocp-ci.dev" "10.132.0.46:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.30:45344 10.132.1.30:443 10.132.0.17:36810 - v1-completions-path [2026-10-30T05:34:06.315Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 62 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "37a54790-9c21-4be2-a592-3284d41724c4" "maas.apps.59fbafd611cf6c800ee5.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.1.30:443 10.132.0.17:36824 - v1-completions-path 2026-10-30T05:34:18.547215Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:34:28.835112Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:34:31.175462Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:34:40.845971Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:34:45.696403Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:35:02.419167Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:35:22.443150Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:35:32.751627Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:35:43.108540Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:35:53.533755Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:36:03.973530Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:36:14.323616Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:36:24.705175Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:36:35.002381Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:36:45.352038Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:36:55.586520Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T05:37:05.278368Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T05:37:05.278394Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T05:37:05.278397Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T05:37:05.278401Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-10-30T05:37:05.247Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 401 - - "-" 66 88 30 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "e6226491-82e7-4099-a375-5fb45a5b22dd" "maas.apps.59fbafd611cf6c800ee5.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.1.30:443 10.132.0.17:47058 - v1-completions-path [2026-10-30T05:37:05.315Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 292 77 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "35209349-2eeb-4ca1-8306-36f0226bd8a5" "maas.apps.59fbafd611cf6c800ee5.prod.konflux-ocp-ci.dev" "10.132.0.47:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.1.30:57672 10.132.1.30:443 10.132.0.17:47072 - v1-completions-path