2026-11-03T00:13:10.231788Z info FLAG: --concurrency="0" 2026-11-03T00:13:10.231808Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T00:13:10.231812Z info FLAG: --help="false" 2026-11-03T00:13:10.231815Z info FLAG: --log_as_json="false" 2026-11-03T00:13:10.231817Z info FLAG: --log_caller="" 2026-11-03T00:13:10.231819Z info FLAG: --log_output_level="default:info" 2026-11-03T00:13:10.231821Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T00:13:10.231825Z info FLAG: --log_target="[stdout]" 2026-11-03T00:13:10.231827Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T00:13:10.231829Z info FLAG: --outlierLogPath="" 2026-11-03T00:13:10.231832Z info FLAG: --profiling="true" 2026-11-03T00:13:10.231833Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T00:13:10.231835Z info FLAG: --proxyLogLevel="warning" 2026-11-03T00:13:10.231837Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T00:13:10.231839Z info FLAG: --stsPort="0" 2026-11-03T00:13:10.231841Z info FLAG: --templateFile="" 2026-11-03T00:13:10.231843Z info FLAG: --tokenManagerPlugin="" 2026-11-03T00:13:10.231846Z info FLAG: --vklog="0" 2026-11-03T00:13:10.231848Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T00:13:10.232022Z info Proxy role ips=[10.132.0.188] type=router id=maas-default-gateway-openshift-default-55f85cf5d7-bbl8r.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T00:13:10.232066Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T00:13:10.233061Z info cpu limit detected as 2, setting concurrency 2026-11-03T00:13:10.233603Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T00:13:10.233614Z info JWT policy is third-party-jwt 2026-11-03T00:13:10.233617Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T00:13:10.235009Z info Starting default Istio SDS Server 2026-11-03T00:13:10.235033Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T00:13:10.235052Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T00:13:10.235465Z info Opening status port 15020 2026-11-03T00:13:10.236631Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T00:13:10.237258Z info sds Starting SDS grpc server 2026-11-03T00:13:10.237346Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T00:13:10.238084Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T00:13:10.238828Z info Starting proxy agent 2026-11-03T00:13:10.238858Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T00:13:10.302326Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T00:13:10.379525Z info cache generated new workload certificate resourceName=default latency=142.078702ms ttl=23h59m59.620480008s 2026-11-03T00:13:10.379572Z info cache Root cert has changed, start rotating root cert 2026-11-03T00:13:10.379619Z info cache returned workload trust anchor from cache ttl=23h59m59.620381448s 2026-11-03T00:13:10.461013Z info ads ADS: new connection for node:1 2026-11-03T00:13:10.461092Z info cache returned workload certificate from cache ttl=23h59m59.538910991s 2026-11-03T00:13:10.461617Z info ads ADS: new connection for node:2 2026-11-03T00:13:10.461693Z info cache returned workload trust anchor from cache ttl=23h59m59.538308004s 2026-11-03T00:13:10.539896Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:13:18.885234Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:13:19.783161Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T00:13:20.810069Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T00:13:21.147697Z info Readiness succeeded in 10.918493043s 2026-11-03T00:13:21.148143Z info Envoy proxy is ready 2026-11-03T00:13:34.484355Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:13:39.100041Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:13:55.437974Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:14:07.161117Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:14:09.599615Z info ads ADS: new connection for node:3 2026-11-03T00:14:09.599786Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T00:14:09.599810Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T00:14:13.631242Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:14:53.148053Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T00:14:57.135Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 328 104 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "49661461-047e-4af5-a87c-b25633eeec5d" "maas.apps.ad05c5333c701bf13b6d.prod.konflux-ocp-ci.dev" "10.134.1.169:8000" outbound|54321||e2e-prio-pool-be38a09f-inference-pool-ip-66ce77f8.llm.svc.cluster.local 10.132.0.188:35918 10.132.0.188:443 10.132.0.13:34648 - v1-model-routing [2026-11-03T00:14:58.011Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 332 58 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "0a778090-d9d9-43bc-983c-3bee79c05420" "maas.apps.ad05c5333c701bf13b6d.prod.konflux-ocp-ci.dev" "10.134.1.169:8000" outbound|54321||e2e-prio-pool-be38a09f-inference-pool-ip-66ce77f8.llm.svc.cluster.local 10.132.0.188:35918 10.132.0.188:443 10.132.0.13:34658 - v1-model-routing [2026-11-03T00:14:58.689Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 349 61 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "48d1b632-7f9d-45b2-b16c-999ad4cb1eff" "maas.apps.ad05c5333c701bf13b6d.prod.konflux-ocp-ci.dev" "10.134.1.169:8000" outbound|54321||e2e-prio-pool-be38a09f-inference-pool-ip-66ce77f8.llm.svc.cluster.local 10.132.0.188:35918 10.132.0.188:443 10.132.0.13:34660 - v1-model-routing 2026-11-03T00:15:16.935010Z info ads ADS: "" 3 terminated with stream closed 2026-11-03T00:15:17.231114Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:15:23.449547Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:15:39.292285Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:18:18.422598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:18:18.422637Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:18:18.422650Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:18:18.422656Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T00:18:18.362Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 60 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "c8a45081-ad3b-4971-99c4-8cb43d4c261a" "maas.apps.ad05c5333c701bf13b6d.prod.konflux-ocp-ci.dev" "10.133.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.188:48114 10.132.0.188:443 10.132.0.13:43682 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T00:18:36.977589Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:19:04.273541Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:19:32.120709Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:19:47.376978Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:19:59.911229Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:20:11.150270Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:20:25.392905Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:20:28.052674Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:20:28.052722Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:20:28.052738Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T00:20:28.052744Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T00:20:27.921Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 335 131 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "1088888d-9039-405d-9516-de5e4c7deb79" "maas.apps.ad05c5333c701bf13b6d.prod.konflux-ocp-ci.dev" "10.133.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.188:33414 10.132.0.188:443 10.132.0.13:42948 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T00:20:29.315192Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-03T00:20:29.609316Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:20:29.609358Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:20:29.609376Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T00:20:29.609381Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T00:20:29.309Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 60 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "f35ceee2-fc18-4953-a5a0-1dcb38606d8d" "maas.apps.ad05c5333c701bf13b6d.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.188:443 10.132.0.13:42972 - v1-completions-path [2026-11-03T00:20:29.556Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 524 53 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "5ab7c1c7-6f25-4341-ae80-bb7badc6f8f5" "maas.apps.ad05c5333c701bf13b6d.prod.konflux-ocp-ci.dev" "10.133.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.188:33420 10.132.0.188:443 10.132.0.13:42988 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T00:20:55.204810Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T00:20:58.449Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 81 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "fd48c44f-2e4b-4acc-9471-7d2a30134146" "maas.apps.ad05c5333c701bf13b6d.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.188:46306 10.132.0.188:443 10.132.0.13:40346 - v1-completions-path [2026-11-03T00:21:25.930Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 283 74 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "a3a55f42-ea62-448c-8764-c28fb14d9c71" "maas.apps.ad05c5333c701bf13b6d.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.188:33362 10.132.0.188:443 10.132.0.13:49242 - v1-completions-path 2026-11-03T00:21:43.176573Z info ads ADS: new connection for node:4 2026-11-03T00:21:43.183349Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-03T00:21:43.183415Z info cache read certificate from file resource=file-root:system 2026-11-03T00:21:47.110433Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:21:47.963277Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:21:57.484388Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"179","StreamId":"13838964124389939577"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=24 [2026-11-03T00:21:57.419Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 65 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "f735514b-8421-4ab5-9dfb-e9e0a5a9a965" "maas.apps.ad05c5333c701bf13b6d.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.132.0.188:443 10.132.0.13:53596 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-11-03T00:22:43.157490Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:22:43.157526Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:22:43.157538Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:22:43.157542Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T00:22:43.157724Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T00:22:42.958Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1707 199 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "347b25ae-d642-4235-aad5-d44f47b5cb3e" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.188:56628 10.132.0.188:443 10.132.0.13:38306 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T00:22:54.712249Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-11-03T00:22:54.786638Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:22:54.786671Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:22:54.786681Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T00:22:54.786685Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:22:54.786839Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T00:22:54.649Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1705 137 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "6249a654-02c4-4c9d-972c-5b16b8b57f83" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.188:56628 10.132.0.188:443 10.132.0.13:43340 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T00:23:43.180845Z info ads ADS: "" 4 terminated 2026-11-03T00:23:43.222797Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:23:45.562139Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:23:55.530817Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:24:13.776642Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:24:19.969018Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T00:24:22.869Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 80 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "38ae4b15-1f77-472c-a508-1db61218ad31" "maas.apps.ad05c5333c701bf13b6d.prod.konflux-ocp-ci.dev" "10.132.0.53:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.188:37992 10.132.0.188:443 10.132.0.13:45444 - v1-completions-path 2026-11-03T00:24:40.122220Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:25:09.030659Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T00:25:17.434Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 66 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "f1015634-8499-4c36-ac69-4a120a0b23b8" "maas.apps.ad05c5333c701bf13b6d.prod.konflux-ocp-ci.dev" "10.132.0.53:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.188:37992 10.132.0.188:443 10.132.0.13:43844 - v1-completions-path [2026-11-03T00:25:21.730Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 62 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "906affcf-afca-4398-9a08-2e82c6a296df" "maas.apps.ad05c5333c701bf13b6d.prod.konflux-ocp-ci.dev" "10.132.0.53:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.188:57700 10.132.0.188:443 10.132.0.13:38128 - v1-completions-path 2026-11-03T00:25:39.533625Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:25:50.395972Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:25:56.528768Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T00:26:09.678Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 66 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "d4712ef9-2f28-4998-a431-0aae5bdf4cff" "maas.apps.ad05c5333c701bf13b6d.prod.konflux-ocp-ci.dev" "10.132.0.53:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.188:57700 10.132.0.188:443 10.132.0.13:47310 - v1-completions-path [2026-11-03T00:26:11.750Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 67 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "5fd2eb80-9d4c-4e9b-9e6b-6c76d4abdbd2" "maas.apps.ad05c5333c701bf13b6d.prod.konflux-ocp-ci.dev" "10.132.0.53:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.188:57700 10.132.0.188:443 10.132.0.13:41336 - v1-completions-path 2026-11-03T00:26:25.297868Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:26:29.805484Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:26:42.687047Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:26:51.399355Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:27:11.884853Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:27:37.648565Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:27:50.721186Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T00:27:53.766Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 66 18 60 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "cf8e5618-405d-43a7-9e89-42f857caedb7" "maas.apps.ad05c5333c701bf13b6d.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.188:443 10.132.0.13:34214 - v1-completions-path 2026-11-03T00:27:56.645164Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:28:14.442168Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:28:16.899224Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:28:16.899265Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:28:16.899280Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:28:16.899285Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T00:28:16.785Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 89 331 113 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "d4bf7a0f-d131-4cef-8340-9c4596b15090" "maas.apps.ad05c5333c701bf13b6d.prod.konflux-ocp-ci.dev" "10.133.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.188:52856 10.132.0.188:443 10.132.0.13:43934 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T00:28:19.867928Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:28:32.437600Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:29:00.962180Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:29:11.738694Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T00:29:28.627Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 39 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "b3be7207-f011-4f42-8344-d9bfa31c86cf" "maas.apps.ad05c5333c701bf13b6d.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.188:443 10.132.0.13:42240 - v1-completions-path 2026-11-03T00:29:32.702035Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:29:39.932163Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:32:03.933812Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:33:32.695004Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:33:50.071082Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:33:55.055664Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:34:08.368645Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:34:24.103677Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:34:32.142824Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:34:35.056788Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:34:35.056832Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:34:35.056850Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:34:35.056855Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T00:34:34.920Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 72 314 136 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "5459e0d0-4d28-4bae-bdd6-b595a07f6932" "maas.apps.ad05c5333c701bf13b6d.prod.konflux-ocp-ci.dev" "10.133.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.188:46756 10.132.0.188:443 10.132.0.13:53944 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T00:34:54.407573Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:35:15.048393Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:35:21.732476Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:35:33.900554Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:35:33.900599Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:35:33.900616Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T00:35:33.900621Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T00:35:33.780Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 67 309 120 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "3a29de10-641b-4d9b-bda1-2400e7d319b2" "maas.apps.ad05c5333c701bf13b6d.prod.konflux-ocp-ci.dev" "10.133.0.132:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.188:52350 10.132.0.188:443 10.132.0.13:51698 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T00:35:50.580532Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:35:56.953537Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:36:03.124346Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T00:36:12.024Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 63 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "8e7c17b8-41a6-4c53-a8f2-87bf0cdfdcbb" "maas.apps.ad05c5333c701bf13b6d.prod.konflux-ocp-ci.dev" "10.132.0.53:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.188:57700 10.132.0.188:443 10.132.0.13:60294 - v1-completions-path 2026-11-03T00:36:35.361010Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:36:46.001489Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:37:01.618787Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:37:09.573527Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:37:16.102927Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T00:37:24.417Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 66 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "bc9fd118-5b61-4e17-8b5b-3cbba53293df" "maas.apps.ad05c5333c701bf13b6d.prod.konflux-ocp-ci.dev" "10.132.0.53:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.188:57700 10.132.0.188:443 10.132.0.13:56762 - v1-completions-path 2026-11-03T00:37:39.443856Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:37:50.042053Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:37:55.031734Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:38:13.205540Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:38:19.553662Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:38:43.099007Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:39:03.293107Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:39:15.860023Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:39:35.763513Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:39:46.471957Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:39:57.478043Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:40:08.161324Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:40:18.848161Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:40:29.742430Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:40:40.286776Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T00:40:51.297Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 288 79 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "142cd590-bb99-474a-97a8-eda3ee89428d" "maas.apps.ad05c5333c701bf13b6d.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.188:46306 10.132.0.188:443 10.132.0.13:38770 - v1-completions-path