2026-11-03T21:14:36.962378Z info FLAG: --concurrency="0" 2026-11-03T21:14:36.962406Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T21:14:36.962410Z info FLAG: --help="false" 2026-11-03T21:14:36.962412Z info FLAG: --log_as_json="false" 2026-11-03T21:14:36.962415Z info FLAG: --log_caller="" 2026-11-03T21:14:36.962416Z info FLAG: --log_output_level="default:info" 2026-11-03T21:14:36.962418Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T21:14:36.962423Z info FLAG: --log_target="[stdout]" 2026-11-03T21:14:36.962427Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T21:14:36.962429Z info FLAG: --outlierLogPath="" 2026-11-03T21:14:36.962431Z info FLAG: --profiling="true" 2026-11-03T21:14:36.962433Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T21:14:36.962435Z info FLAG: --proxyLogLevel="warning" 2026-11-03T21:14:36.962437Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T21:14:36.962439Z info FLAG: --stsPort="0" 2026-11-03T21:14:36.962441Z info FLAG: --templateFile="" 2026-11-03T21:14:36.962443Z info FLAG: --tokenManagerPlugin="" 2026-11-03T21:14:36.962446Z info FLAG: --vklog="0" 2026-11-03T21:14:36.962448Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T21:14:36.962642Z info Proxy role ips=[10.134.1.99] type=router id=maas-default-gateway-openshift-default-57f78bcd55-f4qjp.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T21:14:36.962692Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T21:14:36.963998Z info cpu limit detected as 2, setting concurrency 2026-11-03T21:14:36.964324Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T21:14:36.964338Z info JWT policy is third-party-jwt 2026-11-03T21:14:36.964342Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T21:14:36.966375Z info Starting default Istio SDS Server 2026-11-03T21:14:36.966395Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T21:14:36.966412Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T21:14:36.966597Z info Opening status port 15020 2026-11-03T21:14:36.967480Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T21:14:36.968631Z info sds Starting SDS grpc server 2026-11-03T21:14:36.968762Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T21:14:36.968881Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T21:14:36.969877Z info Starting proxy agent 2026-11-03T21:14:36.969910Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T21:14:37.034989Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T21:14:37.201699Z info ads ADS: new connection for node:1 2026-11-03T21:14:37.202146Z info ads ADS: new connection for node:2 2026-11-03T21:14:37.256009Z info cache generated new workload certificate resourceName=default latency=287.148276ms ttl=23h59m59.743997028s 2026-11-03T21:14:37.256059Z info cache Root cert has changed, start rotating root cert 2026-11-03T21:14:37.256119Z info cache returned workload trust anchor from cache ttl=23h59m59.743882555s 2026-11-03T21:14:37.256138Z info cache returned workload certificate from cache ttl=23h59m59.743863277s 2026-11-03T21:14:37.256240Z info cache returned workload trust anchor from cache ttl=23h59m59.74376585s 2026-11-03T21:14:37.256606Z info cache returned workload trust anchor from cache ttl=23h59m59.743395551s 2026-11-03T21:14:37.368970Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:14:45.906278Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:14:46.815592Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T21:14:47.423426Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T21:14:48.506255Z info Readiness succeeded in 11.547111828s 2026-11-03T21:14:48.506502Z info Readiness succeeded in 11.547367784s 2026-11-03T21:14:48.506661Z info Envoy proxy is ready 2026-11-03T21:15:19.392826Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:15:39.294853Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:15:44.327113Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:15:53.466772Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:15:59.853Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 429 - - "-" 61 18 74 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "48802453-13a3-44d9-b50c-ad0dc05bf819" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.99:443 10.132.0.11:53940 - v1-catch-all-path 2026-11-03T21:16:08.194872Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:16:16.144179Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:16:33.494132Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:16:45.277658Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:16:47.647506Z info ads ADS: new connection for node:3 2026-11-03T21:16:47.647732Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T21:16:47.647760Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T21:16:52.488288Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:17:31.379609Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:17:58.067567Z info ads ADS: "" 3 terminated 2026-11-03T21:17:58.343537Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:18:03.505624Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:18:17.307098Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:21:16.190192Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:21:42.025386Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:22:08.272824Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:22:23.012114Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:22:34.998321Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:22:47.884389Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:22:53.214425Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:22:56.780Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 83 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "73deb3e0-fdd8-42a1-b03d-6513c3544ea5" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.99:54522 10.134.1.99:443 10.132.0.11:34890 - v1-completions-path 2026-11-03T21:23:14.862958Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:23:46.347Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 284 91 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "b88b1176-674e-4332-8645-0e1b63d5c4fb" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.133.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.1.99:47502 10.134.1.99:443 10.132.0.11:58164 - v1-completions-path 2026-11-03T21:24:03.675818Z info ads ADS: new connection for node:4 2026-11-03T21:24:03.683654Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-03T21:24:03.683717Z info cache read certificate from file resource=file-root:system 2026-11-03T21:24:07.765122Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:24:08.890893Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:24:12.012279Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:24:12.012315Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:24:12.012327Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T21:24:12.012331Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T21:24:11.892Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 73 316 119 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "976bc67b-3127-4bdc-8e06-dea901c9ead7" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.99:33742 10.134.1.99:443 10.132.0.11:53894 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:24:16.836586Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"183","StreamId":"13503747262379150025"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=24 [2026-11-03T21:24:16.767Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 69 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "d6206880-635d-472b-b9da-547c22ad66b4" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.134.1.99:443 10.132.0.11:59102 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-11-03T21:24:43.160894Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T21:24:43.160930Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T21:24:43.160972Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-03T21:24:43.160977Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T21:24:43.161152Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-11-03T21:24:42.961Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1533 199 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "31e3d972-b57b-4df6-b3f1-b8a195c7e47b" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.1.99:50984 10.134.1.99:443 10.132.0.11:49180 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T21:24:48.056756Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:24:48.056786Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:24:48.056793Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-11-03T21:24:48.056802Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:24:48.056806Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T21:24:48.056991Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T21:24:47.860Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1539 196 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "80d63a3f-436b-4b93-89ed-026aa7f441b5" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.1.99:35534 10.134.1.99:443 10.132.0.11:51912 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T21:25:09.223173Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:25:09.223203Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:25:09.223214Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:25:09.223218Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T21:25:09.223384Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T21:25:09.097Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1540 126 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "7134b815-b605-447e-b031-616350a8ad2d" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.1.99:35534 10.134.1.99:443 10.132.0.11:32810 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T21:25:21.007301Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=26 2026-11-03T21:25:21.007514Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T21:25:21.007532Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T21:25:21.007541Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-03T21:25:21.007545Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T21:25:21.007703Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-11-03T21:25:20.805Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1539 202 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "f5b25cfd-81fc-4729-8d73-35a3ec9f1db2" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.1.99:50984 10.134.1.99:443 10.132.0.11:49896 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T21:25:50.602340Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:25:50.621024Z info ads ADS: "" 4 terminated with stream closed 2026-11-03T21:25:51.172745Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T21:25:51.172770Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T21:25:51.172774Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T21:25:51.172778Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-11-03T21:25:51.136Z] "POST /llm/premium-simulated-simulated-premium/v1/completions HTTP/1.1" 403 - - "-" 74 77 36 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "2c2f85db-30ac-4c63-962a-ba6de001d172" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "-" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.99:443 10.132.0.11:56808 - v1-completions-path 2026-11-03T21:25:53.654613Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:26:04.887641Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:26:07.694368Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:26:07.694515Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:26:07.694576Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:26:07.694623Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T21:26:07.303Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 87 329 391 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "38069060-6f9d-4c0c-bee0-5907d30ddd17" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.99:50690 10.134.1.99:443 10.132.0.11:52922 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T21:26:09.838Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 60 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "66c6b25a-7261-41ee-9c9a-10b948589699" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.99:443 10.132.0.11:52926 - v1-completions-path 2026-11-03T21:26:23.879876Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:26:28.805498Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:26:33.381Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 299 67 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "c13f5bc4-41c2-41d8-999b-1838e514d6f2" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.99:54522 10.134.1.99:443 10.132.0.11:42264 - v1-completions-path [2026-11-03T21:26:33.643Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 61 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "bbd37d19-003e-425d-a006-5397b85c5b31" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.99:54522 10.134.1.99:443 10.132.0.11:42278 - v1-completions-path 2026-11-03T21:26:53.640698Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:27:06.691644Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:27:15.395Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 61 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "91a18c9c-9d56-4f5f-b576-b692df4b626f" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.99:443 10.132.0.11:50892 - v1-completions-path 2026-11-03T21:27:37.790796Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:27:49.111322Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:27:54.088066Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:28:09.492Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 62 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "577b3515-1ce2-4bcc-a283-bf98a2c0df3e" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.99:52888 10.134.1.99:443 10.132.0.11:39992 - v1-completions-path 2026-11-03T21:28:23.574060Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:28:28.393330Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:28:40.999156Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:28:51.543158Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:29:02.740778Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:29:09.697063Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:29:25.002981Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:29:38.279638Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:29:43.774Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 66 18 66 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "c2c5537b-0b25-429d-96fd-5de3e1fb6560" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.99:443 10.132.0.11:51896 - v1-completions-path 2026-11-03T21:29:44.974987Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:29:59.192Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 40 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "b6be1906-b035-4c68-8ee8-e7cf0f6df4e3" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.99:443 10.132.0.11:46910 - v1-completions-path 2026-11-03T21:30:03.733567Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:30:09.581780Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:30:12.570Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 76 0 40 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "40792620-77f5-4922-a91f-375120ff3e60" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.99:443 10.132.0.11:52808 - v1-completions-path 2026-11-03T21:30:21.511881Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:30:47.047610Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:30:58.179846Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:31:18.563902Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:31:26.905818Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:33:52.045415Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:33:57.277221Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:35:05.611459Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:35:05.611503Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:35:05.611522Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:35:05.611529Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T21:35:05.499Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 313 112 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "2ae39d23-30aa-44d2-a759-c2fca85f3b71" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.99:45286 10.134.1.99:443 10.132.0.11:50684 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:35:27.383460Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:35:45.149761Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:35:50.148997Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:36:02.779895Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:36:19.620457Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:36:23.445862Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:36:28.763Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 65 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "8a12f90d-6b58-4368-a5c6-46b24bcf7087" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.99:54522 10.134.1.99:443 10.132.0.11:58562 - v1-completions-path 2026-11-03T21:36:43.269841Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:36:53.944839Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:37:00.825847Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:37:12.307197Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T21:37:12.307238Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T21:37:12.307256Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-03T21:37:12.307262Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 [2026-11-03T21:37:12.187Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 67 310 120 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "4be9f775-5638-4a8a-b1a3-d0e887e36f7e" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.133.1.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.99:49962 10.134.1.99:443 10.132.0.11:44358 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T21:37:15.145Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 61 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "0a3ee9a4-bf9d-4c82-a555-5758842585bd" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.99:52888 10.134.1.99:443 10.132.0.11:44378 - v1-completions-path 2026-11-03T21:37:29.617190Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:37:36.436689Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:37:43.119763Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:37:55.165Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 70 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "eebf0d0b-7092-4438-9c2c-ed6bb332250c" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.99:52888 10.134.1.99:443 10.132.0.11:54590 - v1-completions-path 2026-11-03T21:38:11.550724Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:38:22.188812Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:38:29.150374Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:38:35.935532Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:38:46.965764Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:39:10.345024Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:39:23.501718Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:39:33.865367Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:39:40.692391Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:40:03.571576Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:40:25.582722Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:40:47.686571Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:41:07.342847Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:41:18.063422Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:41:28.694798Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:41:39.635119Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:41:50.442267Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:42:00.991336Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:42:10.523Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 283 74 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "cd100277-b390-4adb-93fb-3fff0c0d6b44" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.133.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.1.99:50826 10.134.1.99:443 10.132.0.11:50566 - v1-completions-path