2026-11-03T19:40:42.456628Z info FLAG: --concurrency="0" 2026-11-03T19:40:42.456650Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T19:40:42.456654Z info FLAG: --help="false" 2026-11-03T19:40:42.456656Z info FLAG: --log_as_json="false" 2026-11-03T19:40:42.456659Z info FLAG: --log_caller="" 2026-11-03T19:40:42.456661Z info FLAG: --log_output_level="default:info" 2026-11-03T19:40:42.456663Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T19:40:42.456667Z info FLAG: --log_target="[stdout]" 2026-11-03T19:40:42.456669Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T19:40:42.456671Z info FLAG: --outlierLogPath="" 2026-11-03T19:40:42.456673Z info FLAG: --profiling="true" 2026-11-03T19:40:42.456675Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T19:40:42.456677Z info FLAG: --proxyLogLevel="warning" 2026-11-03T19:40:42.456680Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T19:40:42.456682Z info FLAG: --stsPort="0" 2026-11-03T19:40:42.456684Z info FLAG: --templateFile="" 2026-11-03T19:40:42.456686Z info FLAG: --tokenManagerPlugin="" 2026-11-03T19:40:42.456688Z info FLAG: --vklog="0" 2026-11-03T19:40:42.456690Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T19:40:42.456841Z info Proxy role ips=[10.133.0.31] type=router id=maas-default-gateway-openshift-default-5db77fbd9f-t5g9v.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T19:40:42.456890Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T19:40:42.457929Z info cpu limit detected as 2, setting concurrency 2026-11-03T19:40:42.458518Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T19:40:42.458529Z info JWT policy is third-party-jwt 2026-11-03T19:40:42.458531Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T19:40:42.459827Z info Starting default Istio SDS Server 2026-11-03T19:40:42.459849Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T19:40:42.459867Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T19:40:42.460279Z info Opening status port 15020 2026-11-03T19:40:42.461217Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T19:40:42.461733Z info sds Starting SDS grpc server 2026-11-03T19:40:42.461767Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T19:40:42.462557Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T19:40:42.463369Z info Starting proxy agent 2026-11-03T19:40:42.463399Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T19:40:42.526166Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T19:40:42.536810Z info cache generated new workload certificate resourceName=default latency=74.977449ms ttl=23h59m59.463193429s 2026-11-03T19:40:42.536852Z info cache Root cert has changed, start rotating root cert 2026-11-03T19:40:42.536892Z info cache returned workload trust anchor from cache ttl=23h59m59.463111184s 2026-11-03T19:40:42.632804Z info ads ADS: new connection for node:1 2026-11-03T19:40:42.632884Z info cache returned workload certificate from cache ttl=23h59m59.367119094s 2026-11-03T19:40:42.632926Z info ads ADS: new connection for node:2 2026-11-03T19:40:42.633000Z info cache returned workload trust anchor from cache ttl=23h59m59.367000904s 2026-11-03T19:40:42.712359Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:40:50.765680Z info Readiness succeeded in 8.311619687s 2026-11-03T19:40:50.766027Z info Envoy proxy is ready 2026-11-03T19:43:37.465667Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T19:43:37.465706Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T19:43:37.465718Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T19:43:37.465723Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T19:43:37.387Z] "DELETE /v1/api-keys/abd4bdd5-abe7-48ef-b9ea-7ca8d029ec95 HTTP/1.1" 200 - via_upstream - "-" 0 321 78 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "23b5581f-5014-4168-bd43-211068c0dd4e" "maas.apps.67cc2d84bceb8ee80679.prod.konflux-ocp-ci.dev" "10.134.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.31:52312 10.133.0.31:443 10.132.0.13:47558 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T19:43:38.863734Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T19:43:38.863769Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T19:43:38.863780Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T19:43:38.863785Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T19:43:38.783Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 316 80 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "c3f9ed5f-0562-454e-81ed-bf35557e00c7" "maas.apps.67cc2d84bceb8ee80679.prod.konflux-ocp-ci.dev" "10.134.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.31:52328 10.133.0.31:443 10.132.0.13:47580 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T19:43:44.848060Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T19:43:44.848094Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T19:43:44.848106Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T19:43:44.848111Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T19:43:44.832Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 91 15 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "6e5bcb3a-6f5b-4fcb-a34f-79fb3005f7d0" "maas.apps.67cc2d84bceb8ee80679.prod.konflux-ocp-ci.dev" "10.134.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.31:52312 10.133.0.31:443 10.132.0.13:50556 - odh-ai-gateway-infra.maas-api-route.4 [2026-11-03T19:43:50.505Z] "POST /maas-api/internal/v1/subscriptions/select HTTP/1.1" 404 NR route_not_found - "-" 56 0 10 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "288794b6-a280-46ee-8bb8-6c2dd4d3008f" "maas.apps.67cc2d84bceb8ee80679.prod.konflux-ocp-ci.dev" "-" - - 10.133.0.31:443 10.132.0.13:50578 - - 2026-11-03T19:43:52.486445Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:43:53.110704Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T19:43:53.110747Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T19:43:53.110767Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T19:43:53.110773Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T19:43:52.998Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 112 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "5aa0f1fb-5a16-4708-8e57-83f371b2977e" "maas.apps.67cc2d84bceb8ee80679.prod.konflux-ocp-ci.dev" "10.134.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.31:52328 10.133.0.31:443 10.132.0.13:58388 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T19:44:01.956087Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:47:31.752873Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:47:41.923810Z error failed scraping envoy metrics: error scraping http://localhost:15090/stats/prometheus: Get "http://localhost:15090/stats/prometheus": context deadline exceeded 2026-11-03T19:47:43.382727Z warn xdsproxy sendDownstreamDelta took 11.851258776s 2026-11-03T19:48:11.708997Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:48:43.911230Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:49:50.756827Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:50:30.729623Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:51:56.610441Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:52:21.147507Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:54:33.405821Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:56:26.658398Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:09:02.618155Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:09:26.096998Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:10:13.020769Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:10:28.458765Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:10:30.148070Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T20:10:35.415Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 61 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "045885fb-0b49-4c62-b369-2ee01dea5c3f" "maas.apps.67cc2d84bceb8ee80679.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.31:443 10.132.0.13:56394 - v1-completions-path 2026-11-03T20:10:47.256764Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:10:48.968410Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:10:52.106090Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-03T20:10:52.173985Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:10:59.924377Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T20:11:07.415Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 70 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "4e6ffa55-3d4b-41d8-8417-c993434ec8f6" "maas.apps.67cc2d84bceb8ee80679.prod.konflux-ocp-ci.dev" "10.134.0.35:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.31:56240 10.133.0.31:443 10.132.0.13:35176 - v1-completions-path [2026-11-03T20:11:08.733Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 60 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "adfe0768-afa8-4f33-b98a-15aa96123f90" "maas.apps.67cc2d84bceb8ee80679.prod.konflux-ocp-ci.dev" "10.134.0.35:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.31:56248 10.133.0.31:443 10.132.0.13:35186 - v1-completions-path [2026-11-03T20:11:09.426Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 299 59 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "33b28fcc-b472-440e-a66d-f239141d7339" "maas.apps.67cc2d84bceb8ee80679.prod.konflux-ocp-ci.dev" "10.134.0.35:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.31:56248 10.133.0.31:443 10.132.0.13:35188 - v1-completions-path 2026-11-03T20:11:24.199249Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:11:34.426311Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:11:36.125027Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T20:11:42.345Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 61 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "c74c6d5a-6ee6-4543-a900-8eeabda5a762" "maas.apps.67cc2d84bceb8ee80679.prod.konflux-ocp-ci.dev" "10.134.0.35:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.31:56240 10.133.0.31:443 10.132.0.13:55882 - v1-completions-path 2026-11-03T20:11:43.110591Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-03T20:11:43.339767Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-03T20:11:43.053Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 57 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "b23b228a-e0a3-4ff4-9088-168c1a9e62ca" "maas.apps.67cc2d84bceb8ee80679.prod.konflux-ocp-ci.dev" "10.134.0.35:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.31:56240 10.133.0.31:443 10.132.0.13:55888 - v1-completions-path [2026-11-03T20:11:43.284Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 56 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "67c45ee5-adb4-4591-aa7f-cd2dfa67af16" "maas.apps.67cc2d84bceb8ee80679.prod.konflux-ocp-ci.dev" "10.134.0.35:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.31:56248 10.133.0.31:443 10.132.0.13:55898 - v1-completions-path 2026-11-03T20:11:55.156470Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:11:56.734800Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:12:06.201322Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=3 2026-11-03T20:12:13.209132Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:12:15.328863Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:12:31.918028Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:12:33.548699Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:12:49.716831Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:13:00.225227Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:13:02.891272Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:13:21.531060Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T20:13:21.531097Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T20:13:21.531108Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T20:13:21.531113Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T20:13:21.474Z] "DELETE /v1/api-keys/c8227bda-09dc-4cc7-a6a6-04137fc73d65 HTTP/1.1" 404 - via_upstream - "-" 0 29 56 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "fc4d1ecd-b517-45a7-87ac-af5a2cd9c1bc" "maas.apps.67cc2d84bceb8ee80679.prod.konflux-ocp-ci.dev" "10.134.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.31:50220 10.133.0.31:443 10.132.0.13:47354 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T20:13:25.436271Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:13:43.546933Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:13:53.778086Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:17:07.958649Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T20:17:10.644Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 73 296 77 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "a7ae1b00-abd0-4662-8e5d-6b57722b060d" "maas.apps.67cc2d84bceb8ee80679.prod.konflux-ocp-ci.dev" "10.133.0.26:8000" outbound|8000||e2e-trlp-test-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.31:55458 10.133.0.31:443 10.132.0.13:44508 - v1-completions-path 2026-11-03T20:17:22.913273Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:17:24.521225Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:17:41.674649Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:19:07.573481Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:19:27.311804Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:19:39.978789Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:19:51.600075Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:19:55.025426Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:20:03.892518Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:20:06.317963Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T20:20:12.301Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 66 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "710d2a6c-9ee6-424d-9d28-ca5c71768c6f" "maas.apps.67cc2d84bceb8ee80679.prod.konflux-ocp-ci.dev" "10.134.0.35:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.31:56248 10.133.0.31:443 10.132.0.13:59054 - v1-completions-path [2026-11-03T20:20:12.422Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 60 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "9085915b-3fb2-4691-a0f1-d3b56bc8c5d0" "maas.apps.67cc2d84bceb8ee80679.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.31:443 10.132.0.13:50542 - v1-completions-path 2026-11-03T20:20:24.180268Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:23:15.076001Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:23:20.173714Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:23:23.012395Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 2026-11-03T20:23:23.234087Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-11-03T20:23:22.936Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 76 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "84854eaf-10c8-47d1-b130-76571d54d976" "maas.apps.67cc2d84bceb8ee80679.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.31:58214 10.133.0.31:443 10.132.0.13:46336 - v1-catch-all-path [2026-11-03T20:23:23.170Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 63 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "54dbc94a-eb1b-97b7-bb60-820d838b709e" "maas.apps.67cc2d84bceb8ee80679.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.31:58224 10.133.0.31:443 10.132.0.13:46352 - v1-catch-all-path 2026-11-03T20:23:35.403599Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:23:36.916107Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:23:51.256548Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:24:02.027509Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8