2026-11-03T21:14:36.895379Z info FLAG: --concurrency="0" 2026-11-03T21:14:36.895399Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T21:14:36.895405Z info FLAG: --help="false" 2026-11-03T21:14:36.895408Z info FLAG: --log_as_json="false" 2026-11-03T21:14:36.895410Z info FLAG: --log_caller="" 2026-11-03T21:14:36.895414Z info FLAG: --log_output_level="default:info" 2026-11-03T21:14:36.895416Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T21:14:36.895423Z info FLAG: --log_target="[stdout]" 2026-11-03T21:14:36.895426Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T21:14:36.895430Z info FLAG: --outlierLogPath="" 2026-11-03T21:14:36.895433Z info FLAG: --profiling="true" 2026-11-03T21:14:36.895436Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T21:14:36.895439Z info FLAG: --proxyLogLevel="warning" 2026-11-03T21:14:36.895441Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T21:14:36.895444Z info FLAG: --stsPort="0" 2026-11-03T21:14:36.895447Z info FLAG: --templateFile="" 2026-11-03T21:14:36.895450Z info FLAG: --tokenManagerPlugin="" 2026-11-03T21:14:36.895454Z info FLAG: --vklog="0" 2026-11-03T21:14:36.895457Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T21:14:36.895631Z info Proxy role ips=[10.134.1.98] type=router id=maas-default-gateway-openshift-default-57f78bcd55-wf4lm.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T21:14:36.895692Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T21:14:36.896899Z info cpu limit detected as 2, setting concurrency 2026-11-03T21:14:36.897160Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T21:14:36.897174Z info JWT policy is third-party-jwt 2026-11-03T21:14:36.897178Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T21:14:36.898970Z info Starting default Istio SDS Server 2026-11-03T21:14:36.898995Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T21:14:36.899030Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T21:14:36.899306Z info Opening status port 15020 2026-11-03T21:14:36.900153Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T21:14:36.901076Z info sds Starting SDS grpc server 2026-11-03T21:14:36.901105Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T21:14:36.901675Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T21:14:36.902725Z info Starting proxy agent 2026-11-03T21:14:36.902811Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T21:14:36.969319Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T21:14:37.140427Z info cache generated new workload certificate resourceName=default latency=239.21474ms ttl=23h59m59.859578645s 2026-11-03T21:14:37.140478Z info cache Root cert has changed, start rotating root cert 2026-11-03T21:14:37.140526Z info cache returned workload trust anchor from cache ttl=23h59m59.859475049s 2026-11-03T21:14:37.145645Z info ads ADS: new connection for node:1 2026-11-03T21:14:37.145721Z info cache returned workload certificate from cache ttl=23h59m59.8542805s 2026-11-03T21:14:37.146420Z info ads ADS: new connection for node:2 2026-11-03T21:14:37.146549Z info cache returned workload trust anchor from cache ttl=23h59m59.853453667s 2026-11-03T21:14:37.237229Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:14:45.530292Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:14:46.406879Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T21:14:47.378101Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T21:14:48.173508Z info Readiness succeeded in 11.281488784s 2026-11-03T21:14:48.173960Z info Envoy proxy is ready 2026-11-03T21:15:19.375439Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:15:39.275025Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:15:44.325092Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:15:53.444912Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:15:58.765774Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-11-03T21:15:58.679Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 86 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "2233ed9c-8e5d-43d3-9393-ab76c934e7b8" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.38:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.1.98:48266 10.134.1.98:443 10.132.0.11:57120 - v1-catch-all-path 2026-11-03T21:15:59.536669Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-11-03T21:15:59.472Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 64 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "31031315-5dd9-40fd-9950-09c4cf160e5f" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.38:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.1.98:48266 10.134.1.98:443 10.132.0.11:57132 - v1-catch-all-path 2026-11-03T21:16:08.198735Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:16:16.130571Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:16:19.554313Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-11-03T21:16:19.468Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 85 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "7939afe8-bf3d-4d9d-b91b-d433b2a13772" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.38:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.1.98:46560 10.134.1.98:443 10.132.0.11:46602 - v1-catch-all-path 2026-11-03T21:16:33.493452Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:16:45.274099Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:16:47.652790Z info ads ADS: new connection for node:3 2026-11-03T21:16:47.652982Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T21:16:47.653016Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T21:16:52.488156Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:17:31.373016Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:17:58.070603Z info ads ADS: "" 3 terminated 2026-11-03T21:17:58.375641Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:18:03.499879Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:18:17.345102Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:21:05.873031Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:21:05.873060Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:21:05.873071Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:21:05.873076Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T21:21:05.808Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 64 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "ae403c13-88bf-44d8-8ffa-78319ee14dc0" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.98:36372 10.134.1.98:443 10.132.0.11:34892 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:21:16.193792Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:21:42.029777Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:22:05.583051Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:22:05.583088Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:22:05.583103Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T21:22:05.583108Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T21:22:05.524Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 628 58 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "4b695aaa-42bf-4741-af75-6927e3f6e60e" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.98:34370 10.134.1.98:443 10.132.0.11:46836 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:22:08.276339Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:22:23.010037Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:22:35.006208Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:22:47.878872Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:22:53.226721Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:23:00.176383Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-11-03T21:23:00.169Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 77 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "45e48084-6665-4ca6-8977-62832c59fbad" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.98:60634 10.134.1.98:443 10.132.0.11:35126 - v1-completions-path 2026-11-03T21:23:14.864780Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:23:16.877250Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:23:16.877294Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:23:16.877312Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:23:16.877319Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T21:23:16.768Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 88 330 108 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "cf897a85-d122-4f66-b938-92ec1b31c47c" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.98:37308 10.134.1.98:443 10.132.0.11:39186 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T21:24:01.866Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 283 95 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "c1bd4c93-8eda-432f-80ae-2bc3cd34ddba" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.133.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.1.98:55118 10.134.1.98:443 10.132.0.11:43700 - v1-completions-path 2026-11-03T21:24:03.686831Z info ads ADS: new connection for node:4 2026-11-03T21:24:03.692237Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-03T21:24:03.692290Z info cache read certificate from file resource=file-root:system 2026-11-03T21:24:07.734202Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:24:08.922678Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:25:32.958349Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:25:32.958385Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:25:32.958400Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T21:25:32.958404Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:25:32.958574Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T21:25:32.759Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1539 198 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "0b6d8ed1-4f1e-4ea4-b62b-8e3f5e30bae0" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.1.98:57982 10.134.1.98:443 10.132.0.11:46878 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T21:25:50.561032Z info ads ADS: "" 4 terminated with stream closed 2026-11-03T21:25:50.617403Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:25:53.687123Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:25:55.872291Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T21:25:55.872320Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T21:25:55.872323Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T21:25:55.872326Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-11-03T21:25:55.815Z] "POST /llm/premium-simulated-simulated-premium/v1/completions HTTP/1.1" 403 - - "-" 74 77 56 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "1c29261a-b0ca-4fa8-beaf-ce118d6a0499" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "-" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.98:443 10.132.0.11:39540 - v1-completions-path 2026-11-03T21:26:04.887386Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:26:08.332Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 67 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "3f25bb8c-9bc9-4a68-9811-423fa8aef643" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.98:60634 10.134.1.98:443 10.132.0.11:37008 - v1-completions-path [2026-11-03T21:26:08.723Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 62 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "8227687e-c3e1-418b-982c-81159d8b49f9" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.98:60634 10.134.1.98:443 10.132.0.11:37020 - v1-completions-path 2026-11-03T21:26:23.880041Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:26:28.801277Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:26:34.779Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 71 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "62732f15-28ec-4b13-918a-edcc574e29de" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.98:56658 10.134.1.98:443 10.132.0.11:45132 - v1-completions-path 2026-11-03T21:26:53.632698Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:27:06.693356Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:27:13.277484Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:27:13.277525Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:27:13.277549Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:27:13.277557Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T21:27:13.164Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 77 319 112 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "f9ad0bfa-61ed-4054-a94f-909ac0074f43" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.98:60104 10.134.1.98:443 10.132.0.11:43782 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T21:27:16.499Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 66 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "3533407b-7e7f-4b72-9e22-cea6bc982e3c" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.98:60634 10.134.1.98:443 10.132.0.11:46608 - v1-completions-path 2026-11-03T21:27:37.797152Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:27:49.101036Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:27:54.094841Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:28:06.080490Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-03T21:28:06.017Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 63 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "6adaa348-17af-4984-9689-44f635f5762b" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.98:56658 10.134.1.98:443 10.132.0.11:50070 - v1-completions-path 2026-11-03T21:28:23.574347Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:28:28.396189Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:28:40.999471Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:28:51.529646Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:29:02.752589Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:29:09.707130Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:29:24.976087Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:29:38.296592Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:29:44.991668Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:30:03.738118Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:30:09.581034Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:30:10.547Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 36 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "525fdcee-17ed-42b8-bd71-1f4d60ed75c1" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.98:443 10.132.0.11:45920 - v1-completions-path 2026-11-03T21:30:21.499845Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:30:47.081615Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:30:58.173380Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:31:14.016Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 57 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "2063cabd-ce1d-4661-9b30-a0ac9ea5eda0" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.98:443 10.132.0.11:55090 - v1-completions-path 2026-11-03T21:31:18.547062Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:31:26.934152Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:33:52.063327Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:33:57.274839Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:35:27.385441Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:35:45.169310Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:35:50.153289Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:36:02.780298Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:36:19.625087Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:36:23.446114Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:36:43.240072Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:36:53.949547Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:37:00.829480Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:37:13.773Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 63 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "a73a7cc3-5e53-4861-acbc-27b7997eb3fb" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.98:60634 10.134.1.98:443 10.132.0.11:44046 - v1-completions-path 2026-11-03T21:37:15.520257Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:37:15.520303Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:37:15.520322Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:37:15.520329Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T21:37:15.411Z] "DELETE /v1/api-keys/dcdd79e3-6628-4f1b-99e9-79c5df755c51 HTTP/1.1" 200 - via_upstream - "-" 0 365 109 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "21ec0184-5caa-49e9-b1c8-6aad9504e821" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.133.1.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.98:58210 10.134.1.98:443 10.132.0.11:44048 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:37:29.588499Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:37:36.436343Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:37:43.117299Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:37:47.679Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 67 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "987c89d2-dd40-44c0-b519-5c82a88101e8" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.98:56658 10.134.1.98:443 10.132.0.11:59526 - v1-completions-path [2026-11-03T21:37:48.608Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 63 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "e4b95cb5-c4ab-4ad5-a628-72b33623258d" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.98:60634 10.134.1.98:443 10.132.0.11:59532 - v1-completions-path [2026-11-03T21:37:49.710Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 57 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "c77ab71b-e2ed-4496-b56d-397b6b514017" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.98:443 10.132.0.11:59534 - v1-completions-path [2026-11-03T21:37:52.212Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 66 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "c6d15d7b-553e-4b7e-a0b0-7fd604da73ca" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.98:56658 10.134.1.98:443 10.132.0.11:59564 - v1-completions-path [2026-11-03T21:37:52.574Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 67 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "eb370570-fd35-4258-aa5a-45e427368023" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.98:60634 10.134.1.98:443 10.132.0.11:59574 - v1-completions-path 2026-11-03T21:38:11.534582Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:38:22.212034Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:38:29.145034Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:38:35.943834Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:38:46.965623Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:38:54.018Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 67 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "4131ee23-edc1-4edb-8358-8d9668139e88" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.98:60634 10.134.1.98:443 10.132.0.11:51484 - v1-completions-path [2026-11-03T21:38:54.754Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 65 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "d6f395c4-691e-48f1-872f-07a83fd0c8fe" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.98:60634 10.134.1.98:443 10.132.0.11:51494 - v1-completions-path 2026-11-03T21:39:10.365925Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:39:23.470208Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:39:33.844842Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:39:40.689078Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:40:03.566407Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:40:25.565854Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:40:47.696739Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:41:07.351180Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:41:18.075431Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:41:28.696466Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:41:39.635682Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:41:50.430010Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:42:00.988953Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8