2026-11-04T01:33:48.256129Z info FLAG: --concurrency="0" 2026-11-04T01:33:48.256153Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-04T01:33:48.256159Z info FLAG: --help="false" 2026-11-04T01:33:48.256163Z info FLAG: --log_as_json="false" 2026-11-04T01:33:48.256165Z info FLAG: --log_caller="" 2026-11-04T01:33:48.256167Z info FLAG: --log_output_level="default:info" 2026-11-04T01:33:48.256169Z info FLAG: --log_stacktrace_level="default:none" 2026-11-04T01:33:48.256174Z info FLAG: --log_target="[stdout]" 2026-11-04T01:33:48.256176Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-04T01:33:48.256178Z info FLAG: --outlierLogPath="" 2026-11-04T01:33:48.256180Z info FLAG: --profiling="true" 2026-11-04T01:33:48.256182Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-04T01:33:48.256184Z info FLAG: --proxyLogLevel="warning" 2026-11-04T01:33:48.256186Z info FLAG: --serviceCluster="istio-proxy" 2026-11-04T01:33:48.256188Z info FLAG: --stsPort="0" 2026-11-04T01:33:48.256190Z info FLAG: --templateFile="" 2026-11-04T01:33:48.256192Z info FLAG: --tokenManagerPlugin="" 2026-11-04T01:33:48.256196Z info FLAG: --vklog="0" 2026-11-04T01:33:48.256199Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-04T01:33:48.256349Z info Proxy role ips=[10.133.0.187] type=router id=maas-default-gateway-openshift-default-5d84bc6b95-x69t9.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-04T01:33:48.256403Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-04T01:33:48.257527Z info cpu limit detected as 2, setting concurrency 2026-11-04T01:33:48.258163Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-04T01:33:48.258174Z info JWT policy is third-party-jwt 2026-11-04T01:33:48.258177Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-04T01:33:48.259542Z info Starting default Istio SDS Server 2026-11-04T01:33:48.259564Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-04T01:33:48.259582Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-04T01:33:48.260552Z info Opening status port 15020 2026-11-04T01:33:48.260954Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-04T01:33:48.261492Z info sds Starting SDS grpc server 2026-11-04T01:33:48.261507Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-04T01:33:48.262241Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-04T01:33:48.263094Z info Starting proxy agent 2026-11-04T01:33:48.263185Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-04T01:33:48.325746Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-04T01:33:48.368876Z info cache generated new workload certificate resourceName=default latency=107.286186ms ttl=23h59m59.631128076s 2026-11-04T01:33:48.368912Z info cache Root cert has changed, start rotating root cert 2026-11-04T01:33:48.368948Z info cache returned workload trust anchor from cache ttl=23h59m59.631052763s 2026-11-04T01:33:48.469456Z info ads ADS: new connection for node:1 2026-11-04T01:33:48.469517Z info cache returned workload certificate from cache ttl=23h59m59.530484547s 2026-11-04T01:33:48.469646Z info ads ADS: new connection for node:2 2026-11-04T01:33:48.469715Z info cache returned workload trust anchor from cache ttl=23h59m59.530286155s 2026-11-04T01:33:48.551671Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:33:56.544646Z info Readiness succeeded in 8.291065573s 2026-11-04T01:33:56.545025Z info Envoy proxy is ready 2026-11-04T01:33:57.136303Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:34:06.878358Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:34:23.161476Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:34:35.094980Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:34:37.775902Z info ads ADS: new connection for node:3 2026-11-04T01:34:37.776066Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-04T01:34:37.776092Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-04T01:34:40.502031Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:35:21.333617Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:35:26.521Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 326 107 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "a982b747-a9c5-44fb-b994-084c02433024" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.133.0.188:8000" outbound|54321||e2e-prio-pool-9e0c44d6-inference-pool-ip-b76b9531.llm.svc.cluster.local 10.133.0.187:39562 10.133.0.187:443 10.132.0.9:49030 - v1-model-routing [2026-11-04T01:35:27.774Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 326 66 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "5555de72-6e50-4d81-a5fa-c9120b905c70" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.133.0.188:8000" outbound|54321||e2e-prio-pool-9e0c44d6-inference-pool-ip-b76b9531.llm.svc.cluster.local 10.133.0.187:39572 10.133.0.187:443 10.132.0.9:49040 - v1-model-routing 2026-11-04T01:35:45.743348Z info ads ADS: "" 3 terminated with stream closed 2026-11-04T01:35:46.138931Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:35:51.433820Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:36:08.179371Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:38:42.462087Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:39:10.822462Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:39:30.468462Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:39:44.466983Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:39:51.521352Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:40:02.461093Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:40:07.539511Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:40:11.424Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 56 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "d4a8a694-05b3-4081-bfa3-0ae48b77c119" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.187:443 10.132.0.9:36858 - v1-completions-path 2026-11-04T01:40:25.682922Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:40:42.976Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 79 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "78dcd6cb-523b-4c29-8f80-a53bc405b96f" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.132.0.51:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.187:36806 10.133.0.187:443 10.132.0.9:48056 - v1-completions-path 2026-11-04T01:41:13.590368Z info ads ADS: new connection for node:4 2026-11-04T01:41:13.598034Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-04T01:41:13.598119Z info cache read certificate from file resource=file-root:system 2026-11-04T01:41:17.280364Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:41:18.882268Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:41:24.168829Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"144","StreamId":"5920288679139546548"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=25 [2026-11-04T01:41:24.104Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 65 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "8a6c6ede-9b87-4d14-a045-e8741d29a30b" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.133.0.187:443 10.132.0.9:45734 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-11-04T01:41:40.847588Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:41:40.847624Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:41:40.847635Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T01:41:40.847639Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:41:40.847807Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-04T01:41:40.643Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1529 204 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "fbaf154c-c099-4263-b07c-7a5e67a193cf" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.187:50512 10.133.0.187:443 10.132.0.9:48944 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-04T01:41:57.495268Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:41:57.495301Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:41:57.495310Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T01:41:57.495315Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:41:57.495462Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-04T01:41:57.303Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1534 191 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "83869c5a-63f7-4c60-b509-1c20983cbdce" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.187:55578 10.133.0.187:443 10.132.0.9:59744 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-04T01:42:19.040229Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-04T01:42:19.109602Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:42:19.109635Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:42:19.109645Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T01:42:19.109650Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:42:19.109813Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-04T01:42:18.979Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1535 130 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "dc497657-5dae-4c95-a5ad-ecdf4af574d8" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.187:55578 10.133.0.187:443 10.132.0.9:49688 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-04T01:42:28.431831Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-11-04T01:42:28.501934Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:42:28.501975Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:42:28.501992Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T01:42:28.501998Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:42:28.502230Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-04T01:42:28.377Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1528 124 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "b93a7a79-49de-47d5-8e95-8977fdd1ca78" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.187:50512 10.133.0.187:443 10.132.0.9:45982 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-04T01:42:33.192743Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:42:33.192784Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:42:33.192795Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:42:33.192800Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T01:42:33.192955Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-04T01:42:33.066Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1534 126 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "a91d2d44-4af8-4ee8-95db-4fffbe3350c2" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.187:50512 10.133.0.187:443 10.132.0.9:49074 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-04T01:42:55.585216Z info ads ADS: "" 4 terminated 2026-11-04T01:42:57.105002Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:42:58.369838Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:43:09.277453Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:43:27.768709Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:43:32.698699Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:43:57.273434Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:44:10.449014Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:44:20.156Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 81 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "581ff92b-e6cb-4a51-be0b-6d8b9067a7ec" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.187:57380 10.133.0.187:443 10.132.0.9:37754 - v1-completions-path [2026-11-04T01:44:21.527Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 61 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "75ad14a3-927c-4fe9-8a44-ce3840bcbdbc" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.187:57390 10.133.0.187:443 10.132.0.9:40816 - v1-completions-path [2026-11-04T01:44:23.109Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 61 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "2f3d4044-7b4c-4be4-b1c4-eddc3693e025" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.187:57380 10.133.0.187:443 10.132.0.9:40830 - v1-completions-path 2026-11-04T01:44:41.591579Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:44:52.296636Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:44:57.391464Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:45:10.713Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 59 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "8b42c82d-3f68-40ce-9cc4-6d3fd4fd2337" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.187:57380 10.133.0.187:443 10.132.0.9:46556 - v1-completions-path 2026-11-04T01:45:26.977449Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:45:31.937216Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:45:46.036738Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:45:55.469790Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:46:03.582661Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:46:12.919984Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:46:22.671814Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:46:41.194455Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:46:55.344Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 38 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "2c1708df-28cd-4d7e-9b71-04c2186a4099" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.187:443 10.132.0.9:48740 - v1-completions-path 2026-11-04T01:47:12.723369Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:47:22.144Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 62 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "745f8abb-6398-405e-a012-405c0ea22b66" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.187:57380 10.133.0.187:443 10.132.0.9:37588 - v1-completions-path 2026-11-04T01:47:36.132906Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:47:46.847755Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:51:12.826Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 48 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "a7289c99-5278-49e2-978b-c9ef561bde09" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.187:443 10.132.0.9:48642 - v1-completions-path 2026-11-04T01:52:47.804499Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:53:05.032640Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:53:09.614798Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:53:13.020354Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-04T01:53:13.020381Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-04T01:53:13.020384Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-04T01:53:13.020387Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-11-04T01:53:12.984Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 123 36 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "447eb95a-4a9b-46af-a148-1073be60cb89" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.187:443 10.132.0.9:47508 - v1-completions-path 2026-11-04T01:53:22.324979Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:53:39.974333Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:53:46.504838Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:53:49.142067Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:53:49.142122Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:53:49.142137Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T01:53:49.142142Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-04T01:53:49.013Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 72 314 128 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "6dda1af4-33b0-4a85-80b0-62501e2197e2" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.133.0.192:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.187:42852 10.133.0.187:443 10.132.0.9:52426 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T01:53:52.034570Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:53:52.034605Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:53:52.034619Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:53:52.034625Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-04T01:53:51.918Z] "DELETE /v1/api-keys/b7a15844-34a4-4452-abf0-44d6f82ad093 HTTP/1.1" 200 - via_upstream - "-" 0 369 116 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "6d283630-167d-4d00-9357-5e4ac2a7b4f8" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.133.0.192:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.187:42852 10.133.0.187:443 10.132.0.9:47478 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T01:54:05.962799Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:54:16.631142Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:54:22.832118Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:54:34.508Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 75 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "06cd4270-57a0-45e8-ab36-75727a4176e2" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.187:57390 10.133.0.187:443 10.132.0.9:35304 - v1-completions-path [2026-11-04T01:54:36.522Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 59 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "ea0b6179-62e7-444f-8bbd-8619951f689a" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.187:57390 10.133.0.187:443 10.132.0.9:35330 - v1-completions-path 2026-11-04T01:54:50.764137Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:54:56.899779Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:55:02.788112Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:55:11.168Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 64 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "34306d39-324b-4d8e-989f-89b3b47e8ba2" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.187:57390 10.133.0.187:443 10.132.0.9:38546 - v1-completions-path [2026-11-04T01:55:14.340Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 66 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "c42b821e-5e36-4efb-9011-7a1e49a9720c" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.187:57390 10.133.0.187:443 10.132.0.9:38558 - v1-completions-path [2026-11-04T01:55:19.155Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 68 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "674e0654-fdb9-4ad4-a38a-49ce4119a3c3" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.187:57390 10.133.0.187:443 10.132.0.9:38590 - v1-completions-path [2026-11-04T01:55:19.903Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 59 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "553fa2fe-6da7-46c5-93c3-9ca85637c82e" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.187:57390 10.133.0.187:443 10.132.0.9:38606 - v1-completions-path 2026-11-04T01:55:34.465006Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:55:45.270906Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:55:51.709888Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:55:59.244660Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:56:10.450664Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:56:33.837642Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:56:44.662464Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:56:49.416679Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:57:07.810553Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:57:13.891400Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:57:35.961880Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:57:57.828006Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:58:18.156273Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:58:30.653098Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:58:50.671954Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:59:01.581426Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:59:12.308281Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:59:23.153060Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:59:33.755821Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:59:40.065069Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-04T01:59:40.065095Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-04T01:59:40.065098Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-04T01:59:40.065128Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-11-04T01:59:40.041Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 401 - - "-" 66 88 23 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "7bc9d0f8-73e0-4261-88e5-98240f714fee" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.187:443 10.132.0.9:42224 - v1-completions-path [2026-11-04T01:59:45.085Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 283 69 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "573b6573-8788-412d-9a9f-06285ecf69e9" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.132.0.51:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.187:36806 10.133.0.187:443 10.132.0.9:37568 - v1-completions-path