2026-11-02T22:36:28.363880Z info FLAG: --concurrency="0" 2026-11-02T22:36:28.363903Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-02T22:36:28.363907Z info FLAG: --help="false" 2026-11-02T22:36:28.363909Z info FLAG: --log_as_json="false" 2026-11-02T22:36:28.363911Z info FLAG: --log_caller="" 2026-11-02T22:36:28.363913Z info FLAG: --log_output_level="default:info" 2026-11-02T22:36:28.363915Z info FLAG: --log_stacktrace_level="default:none" 2026-11-02T22:36:28.363919Z info FLAG: --log_target="[stdout]" 2026-11-02T22:36:28.363922Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-02T22:36:28.363924Z info FLAG: --outlierLogPath="" 2026-11-02T22:36:28.363926Z info FLAG: --profiling="true" 2026-11-02T22:36:28.363928Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-02T22:36:28.363930Z info FLAG: --proxyLogLevel="warning" 2026-11-02T22:36:28.363932Z info FLAG: --serviceCluster="istio-proxy" 2026-11-02T22:36:28.363934Z info FLAG: --stsPort="0" 2026-11-02T22:36:28.363936Z info FLAG: --templateFile="" 2026-11-02T22:36:28.363938Z info FLAG: --tokenManagerPlugin="" 2026-11-02T22:36:28.363941Z info FLAG: --vklog="0" 2026-11-02T22:36:28.363943Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-02T22:36:28.364134Z info Proxy role ips=[10.134.0.200] type=router id=maas-default-gateway-openshift-default-fcb75c45f-7zm8m.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-02T22:36:28.364187Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-02T22:36:28.365283Z info cpu limit detected as 2, setting concurrency 2026-11-02T22:36:28.365494Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-02T22:36:28.365505Z info JWT policy is third-party-jwt 2026-11-02T22:36:28.365508Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-02T22:36:28.366983Z info Starting default Istio SDS Server 2026-11-02T22:36:28.367002Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-02T22:36:28.367020Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-02T22:36:28.367780Z info Opening status port 15020 2026-11-02T22:36:28.368516Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-02T22:36:28.369593Z info sds Starting SDS grpc server 2026-11-02T22:36:28.369622Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-02T22:36:28.369923Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-02T22:36:28.370997Z info Starting proxy agent 2026-11-02T22:36:28.371041Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-02T22:36:28.439634Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-02T22:36:28.502657Z info cache generated new workload certificate resourceName=default latency=132.895185ms ttl=23h59m59.497347737s 2026-11-02T22:36:28.502700Z info cache Root cert has changed, start rotating root cert 2026-11-02T22:36:28.502737Z info cache returned workload trust anchor from cache ttl=23h59m59.497263792s 2026-11-02T22:36:28.602868Z info ads ADS: new connection for node:1 2026-11-02T22:36:28.602882Z info ads ADS: new connection for node:2 2026-11-02T22:36:28.603388Z info cache returned workload trust anchor from cache ttl=23h59m59.396614005s 2026-11-02T22:36:28.603394Z info cache returned workload certificate from cache ttl=23h59m59.396609288s 2026-11-02T22:36:28.687638Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:36:36.441778Z info Readiness succeeded in 8.080602296s 2026-11-02T22:36:36.442233Z info Envoy proxy is ready 2026-11-02T22:36:42.185631Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:37:17.242350Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T22:37:22.201Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 140 405 85 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "349dd1a3-ec28-41a3-9234-a3dad0ef5ddd" "maas.apps.1f35823228f5b77bf43a.prod.konflux-ocp-ci.dev" "10.133.0.36:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.134.0.200:49198 10.134.0.200:443 10.132.0.20:50474 - v1-chat-completions-path 2026-11-02T22:37:35.916025Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:37:40.845743Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:37:41.487860Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:37:41.487916Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:37:41.487941Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T22:37:41.487968Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-02T22:37:41.358Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 322 129 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "3ca9d6f9-55ef-4e30-a9e9-1cf0dce4d27f" "maas.apps.1f35823228f5b77bf43a.prod.konflux-ocp-ci.dev" "10.132.0.49:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.200:46000 10.134.0.200:443 10.132.0.20:42720 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T22:37:51.252321Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:37:53.579852Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:37:53.579907Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:37:53.579929Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T22:37:53.579937Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-02T22:37:52.028Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 317 1551 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "228d6862-325a-4180-9052-32f9adc781eb" "maas.apps.1f35823228f5b77bf43a.prod.konflux-ocp-ci.dev" "10.132.0.49:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.200:53944 10.134.0.200:443 10.132.0.20:40048 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T22:37:56.536439Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-11-02T22:37:56.467Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 69 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "4994653d-247d-4747-98cf-f7185fdb843a" "maas.apps.1f35823228f5b77bf43a.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.200:56882 10.134.0.200:443 10.132.0.20:56970 - v1-catch-all-path 2026-11-02T22:38:04.906765Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:38:14.265030Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:38:17.239519Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:38:17.239555Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:38:17.239568Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:38:17.239573Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T22:38:17.096Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 73 316 143 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "27371f2e-fbd3-4d25-9393-201e43326335" "maas.apps.1f35823228f5b77bf43a.prod.konflux-ocp-ci.dev" "10.132.0.49:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.200:53944 10.134.0.200:443 10.132.0.20:60390 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T22:38:24.235416Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:38:42.822692Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:38:44.872661Z info ads ADS: new connection for node:3 2026-11-02T22:38:44.872926Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-02T22:38:44.873026Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-02T22:38:48.053076Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:39:28.663697Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:39:51.908860Z info ads ADS: "" 3 terminated 2026-11-02T22:39:52.392417Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:39:57.357909Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:40:10.716593Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:42:44.283775Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:43:08.671042Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:43:28.336487Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:43:42.264348Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:44:00.884077Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:44:13.219452Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:44:18.337940Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:44:39.198179Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:44:41.103833Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:44:41.103870Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:44:41.103884Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:44:41.103889Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-02T22:44:40.988Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 88 331 115 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "d9862cc0-6376-410e-b305-3a7d1bb0c2a7" "maas.apps.1f35823228f5b77bf43a.prod.konflux-ocp-ci.dev" "10.132.0.49:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.200:60760 10.134.0.200:443 10.132.0.20:42026 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T22:45:26.571797Z info ads ADS: new connection for node:4 2026-11-02T22:45:26.576313Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-02T22:45:26.576354Z info cache read certificate from file resource=file-root:system 2026-11-02T22:45:30.218668Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:45:31.689006Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:46:12.311843Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:46:12.311875Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:46:12.311884Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:46:12.311889Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T22:46:12.312087Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-02T22:46:12.116Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1713 196 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "35f43074-5b94-42f4-ab4f-73a52f099508" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.200:52974 10.134.0.200:443 10.132.0.20:42990 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T22:46:34.413771Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:46:34.413810Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:46:34.413817Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-11-02T22:46:34.413827Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:46:34.413832Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T22:46:34.414054Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-02T22:46:34.218Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1711 195 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "a9b77d62-5f04-422f-98be-93f9d88df87d" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.200:41110 10.134.0.200:443 10.132.0.20:43760 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T22:47:00.221355Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-02T22:47:00.340045Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:47:00.340078Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:47:00.340091Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:47:00.340097Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T22:47:00.340286Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-02T22:47:00.214Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1715 126 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "70d28f74-4b95-4566-bf0f-49e43ea0daa1" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.200:52974 10.134.0.200:443 10.132.0.20:41058 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T22:47:17.706658Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:47:17.731028Z info ads ADS: "" 4 terminated with stream closed 2026-11-02T22:47:18.696668Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:47:23.732392Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:47:29.373708Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T22:47:37.378Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 73 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "d67078e5-78bb-4c0a-9c43-23242f1a6207" "maas.apps.1f35823228f5b77bf43a.prod.konflux-ocp-ci.dev" "10.134.0.38:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.200:51488 10.134.0.200:443 10.132.0.20:60724 - v1-completions-path 2026-11-02T22:47:52.153105Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:47:57.033893Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:48:21.386304Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:48:34.464385Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:49:04.989135Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:49:15.710268Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:49:20.546682Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T22:49:33.600Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 58 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "efccf2a3-d0cd-46d1-b262-d3911bf72c36" "maas.apps.1f35823228f5b77bf43a.prod.konflux-ocp-ci.dev" "10.134.0.38:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.200:52880 10.134.0.200:443 10.132.0.20:52824 - v1-completions-path 2026-11-02T22:49:50.295566Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:49:55.046772Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:49:57.913547Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:49:57.913583Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:49:57.913599Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T22:49:57.913604Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-02T22:49:57.798Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 73 315 114 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "ccb576a8-fa5d-4da0-8ef0-6bb3b3d850ce" "maas.apps.1f35823228f5b77bf43a.prod.konflux-ocp-ci.dev" "10.132.0.49:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.200:39848 10.134.0.200:443 10.132.0.20:53728 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T22:50:08.819833Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:50:18.325071Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:50:29.102105Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:50:35.773575Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:50:50.619813Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:51:03.474537Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:51:09.488349Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:51:29.742289Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:51:35.356149Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:51:46.820800Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:51:58.376707Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:51:58.376744Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:51:58.376759Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:51:58.376764Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-02T22:51:58.260Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 67 310 116 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "e9736c7d-afb3-4c34-a0e9-34868375bc97" "maas.apps.1f35823228f5b77bf43a.prod.konflux-ocp-ci.dev" "10.132.0.49:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.200:44570 10.134.0.200:443 10.132.0.20:55288 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T22:51:59.132Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 297 60 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "38c24e87-8b2b-4b13-b0d8-7ed59ac1ae0e" "maas.apps.1f35823228f5b77bf43a.prod.konflux-ocp-ci.dev" "10.134.0.38:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.200:52880 10.134.0.200:443 10.132.0.20:55300 - v1-completions-path 2026-11-02T22:52:12.956485Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:52:23.593504Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:52:42.438230Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:52:47.359220Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:55:21.292046Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:56:50.666265Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:57:07.604254Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:57:12.166205Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:57:23.453876Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:57:41.351805Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:57:47.644446Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T22:57:52.546Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 78 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "b37ecac1-bc22-4b65-8a93-136f8d3423ec" "maas.apps.1f35823228f5b77bf43a.prod.konflux-ocp-ci.dev" "10.134.0.38:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.200:52880 10.134.0.200:443 10.132.0.20:40038 - v1-completions-path 2026-11-02T22:58:07.322548Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:58:17.880333Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:58:23.992964Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T22:58:36.370Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 57 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "472c1d72-5b5f-41a6-8d31-49d1bb81698c" "maas.apps.1f35823228f5b77bf43a.prod.konflux-ocp-ci.dev" "10.134.0.38:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.200:51488 10.134.0.200:443 10.132.0.20:59134 - v1-completions-path 2026-11-02T22:58:37.381026Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:58:37.381059Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:58:37.381074Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T22:58:37.381079Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-02T22:58:36.731Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 52 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "ce696df5-c83d-413e-84e2-7e1efac90c9b" "maas.apps.1f35823228f5b77bf43a.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.200:443 10.132.0.20:59156 - v1-completions-path [2026-11-02T22:58:37.270Z] "DELETE /v1/api-keys/2efbeb7d-64da-4348-9bc7-283f057aa4f9 HTTP/1.1" 200 - via_upstream - "-" 0 365 110 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "cce77bda-0923-4d8b-98e8-575bf87f8af7" "maas.apps.1f35823228f5b77bf43a.prod.konflux-ocp-ci.dev" "10.134.0.242:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.200:35228 10.134.0.200:443 10.132.0.20:59170 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T22:58:51.013344Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:58:57.109266Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:59:02.789021Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T22:59:14.963Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 62 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "a3bfc382-0ba3-4d52-b791-79e05f804ab8" "maas.apps.1f35823228f5b77bf43a.prod.konflux-ocp-ci.dev" "10.134.0.38:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.200:51488 10.134.0.200:443 10.132.0.20:33732 - v1-completions-path [2026-11-02T22:59:16.761Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 64 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "6917bb13-ebdd-4d5a-b73c-cc2724fc52ed" "maas.apps.1f35823228f5b77bf43a.prod.konflux-ocp-ci.dev" "10.134.0.38:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.200:52880 10.134.0.200:443 10.132.0.20:33758 - v1-completions-path [2026-11-02T22:59:18.206Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 65 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "31f0dcca-0c14-43d3-9c8d-38fda64e1c4f" "maas.apps.1f35823228f5b77bf43a.prod.konflux-ocp-ci.dev" "10.134.0.38:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.200:51488 10.134.0.200:443 10.132.0.20:33772 - v1-completions-path 2026-11-02T22:59:33.360939Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:59:44.262437Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:59:50.550291Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:59:57.727878Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T23:00:08.545207Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T23:00:31.998036Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T23:00:42.615534Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T23:00:47.203252Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T23:01:05.232727Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T23:01:11.757344Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T23:01:31.776705Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T23:01:53.349378Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T23:02:15.429019Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T23:02:35.473843Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T23:02:46.376676Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T23:02:57.113407Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T23:03:07.833095Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T23:03:18.668947Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T23:03:29.283109Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T23:04:01.924113Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2