2026-10-31T17:24:37.227141Z info FLAG: --concurrency="0" 2026-10-31T17:24:37.227162Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-31T17:24:37.227166Z info FLAG: --help="false" 2026-10-31T17:24:37.227168Z info FLAG: --log_as_json="false" 2026-10-31T17:24:37.227170Z info FLAG: --log_caller="" 2026-10-31T17:24:37.227173Z info FLAG: --log_output_level="default:info" 2026-10-31T17:24:37.227175Z info FLAG: --log_stacktrace_level="default:none" 2026-10-31T17:24:37.227180Z info FLAG: --log_target="[stdout]" 2026-10-31T17:24:37.227182Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-31T17:24:37.227184Z info FLAG: --outlierLogPath="" 2026-10-31T17:24:37.227187Z info FLAG: --profiling="true" 2026-10-31T17:24:37.227189Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-31T17:24:37.227191Z info FLAG: --proxyLogLevel="warning" 2026-10-31T17:24:37.227193Z info FLAG: --serviceCluster="istio-proxy" 2026-10-31T17:24:37.227195Z info FLAG: --stsPort="0" 2026-10-31T17:24:37.227197Z info FLAG: --templateFile="" 2026-10-31T17:24:37.227199Z info FLAG: --tokenManagerPlugin="" 2026-10-31T17:24:37.227206Z info FLAG: --vklog="0" 2026-10-31T17:24:37.227208Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-31T17:24:37.227429Z info Proxy role ips=[10.133.0.27] type=router id=maas-default-gateway-openshift-default-9dccdc9b7-26sm5.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-31T17:24:37.227478Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-31T17:24:37.228770Z info cpu limit detected as 2, setting concurrency 2026-10-31T17:24:37.228977Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-31T17:24:37.228989Z info JWT policy is third-party-jwt 2026-10-31T17:24:37.228992Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-31T17:24:37.230785Z info Starting default Istio SDS Server 2026-10-31T17:24:37.230807Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-31T17:24:37.230824Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-31T17:24:37.230887Z info Opening status port 15020 2026-10-31T17:24:37.231751Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-31T17:24:37.234196Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-31T17:24:37.235189Z info Starting proxy agent 2026-10-31T17:24:37.235224Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-31T17:24:37.242809Z info sds Starting SDS grpc server 2026-10-31T17:24:37.242825Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-31T17:24:37.300589Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-31T17:24:37.323965Z info cache generated new workload certificate resourceName=default latency=91.458389ms ttl=23h59m59.676039007s 2026-10-31T17:24:37.324001Z info cache Root cert has changed, start rotating root cert 2026-10-31T17:24:37.324037Z info cache returned workload trust anchor from cache ttl=23h59m59.675963405s 2026-10-31T17:24:37.413377Z info ads ADS: new connection for node:1 2026-10-31T17:24:37.413440Z info cache returned workload certificate from cache ttl=23h59m59.586561304s 2026-10-31T17:24:37.414064Z info ads ADS: new connection for node:2 2026-10-31T17:24:37.414139Z info cache returned workload trust anchor from cache ttl=23h59m59.585863574s 2026-10-31T17:24:37.493421Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:24:45.409083Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:24:47.308031Z info Readiness succeeded in 10.083268698s 2026-10-31T17:24:47.308471Z info Envoy proxy is ready 2026-10-31T17:24:47.390531Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:24:47.413449Z info Readiness succeeded in 10.188690713s 2026-10-31T17:25:34.799591Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T17:25:34.799639Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T17:25:34.799675Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T17:25:34.799683Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-31T17:25:34.685Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 320 113 - "52.71.36.72,10.132.0.12" "curl/7.76.1" "4feee49e-e626-4efd-a521-99af399479fa" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "10.132.0.54:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.27:47630 10.133.0.27:443 10.132.0.12:55030 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T17:25:36.138321Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T17:25:36.138348Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T17:25:36.138352Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T17:25:36.138356Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-10-31T17:25:35.306Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 410 74 - "52.71.36.72,10.132.0.12" "curl/7.76.1" "ccac54bf-1581-435a-8712-5a362121061c" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "10.134.0.36:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.27:52116 10.133.0.27:443 10.132.0.12:55046 - v1-model-routing [2026-10-31T17:25:35.682Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 49 - "52.71.36.72,10.132.0.12" "curl/7.76.1" "29b4a588-7d7f-45d2-85f1-1bf63fd76ff1" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.27:443 10.132.0.12:55052 - v1-model-routing [2026-10-31T17:25:36.121Z] "POST /v1/chat/completions HTTP/1.1" 401 - - "-" 122 88 16 - "52.71.36.72,10.132.0.12" "curl/7.76.1" "4ba18a52-8e22-498d-a9a2-173f432126c5" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.27:443 10.132.0.12:55068 - v1-model-routing 2026-10-31T17:28:47.251978Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T17:28:47.252019Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T17:28:47.252038Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T17:28:47.252046Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-31T17:28:47.154Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 84 326 97 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "d7928385-97de-4e68-a60b-1e6c51899dc1" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "10.132.0.54:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.27:53100 10.133.0.27:443 10.132.0.12:60580 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T17:28:49.072930Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T17:28:49.072981Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T17:28:49.073004Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T17:28:49.073011Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-31T17:28:48.942Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 130 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "ef5df44b-23ac-40c6-bf92-49447283c106" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "10.132.0.54:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.27:55398 10.133.0.27:443 10.132.0.12:60592 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T17:28:50.212995Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T17:28:50.213029Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T17:28:50.213043Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T17:28:50.213048Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-31T17:28:50.105Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 312 107 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "aedca54c-f659-465f-b615-31a65532a8a9" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "10.132.0.54:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.27:55398 10.133.0.27:443 10.132.0.12:60596 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T17:28:50.597815Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T17:28:50.597849Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T17:28:50.597862Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T17:28:50.597867Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-31T17:28:50.462Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 135 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "7341dd0d-2e98-4ed9-8244-475191fec710" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "10.132.0.54:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.27:55398 10.133.0.27:443 10.132.0.12:60612 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T17:29:00.262507Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:29:00.326170Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T17:29:00.326206Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T17:29:00.326220Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T17:29:00.326225Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T17:29:01.022180Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T17:29:01.022216Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T17:29:01.022230Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T17:29:01.022235Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-31T17:29:00.183Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 143 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "c7dce610-90fb-4921-a64a-da490516a6fc" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "10.132.0.54:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.27:53100 10.133.0.27:443 10.132.0.12:54536 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T17:29:00.928Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 93 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "e05b5ad8-63af-45f5-905b-1d2c69b4e35b" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "10.132.0.54:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.27:53100 10.133.0.27:443 10.132.0.12:54544 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T17:29:01.063Z] "POST /maas-api/internal/v1/subscriptions/select HTTP/1.1" 404 NR route_not_found - "-" 56 0 7 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "1e72f349-b0a4-4733-84c3-77ca422bc6cf" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "-" - - 10.133.0.27:443 10.132.0.12:54552 - - [2026-10-31T17:29:02.408Z] "POST /maas-api/internal/v1/api-keys/cleanup HTTP/1.1" 404 NR route_not_found - "-" 2 0 10 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "d8509745-4fbf-42c9-a98a-c96eb075d5f8" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "-" - - 10.133.0.27:443 10.132.0.12:33948 - - 2026-10-31T17:29:13.440144Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:33:18.649549Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T17:33:18.649581Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T17:33:18.649594Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T17:33:18.649598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-31T17:33:18.632Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 91 16 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "566c95b7-6c1c-484b-b335-0f6e5e9883f9" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "10.132.0.54:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.27:46936 10.133.0.27:443 10.132.0.12:49280 - odh-ai-gateway-infra.maas-api-route.4 2026-10-31T17:34:02.487775Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:34:32.812430Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:34:52.903940Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:37:06.739906Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:41:00.903008Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:42:58.564482Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:44:31.754532Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:45:40.855815Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:53:27.530270Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-10-31T17:56:57.657156Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-10-31T17:56:57.657250Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T17:56:57.657268Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T17:56:57.657283Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T17:56:57.657288Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-31T17:56:57.554Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 30 314 103 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "72483e2a-0347-4b0c-9ef5-052f6d25025c" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "10.132.0.54:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.27:34828 10.133.0.27:443 10.132.0.12:35826 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T17:57:00.210840Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:57:34.972870Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:58:26.761651Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=3 2026-10-31T17:59:07.156411Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T17:59:11.923Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 71 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "ccb7749d-9054-4b78-a734-eb0c3403d719" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.27:50476 10.133.0.27:443 10.132.0.12:33134 - v1-completions-path 2026-10-31T17:59:34.361367Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:59:48.090308Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:59:52.503579Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-10-31T17:59:52.445Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 60 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "125bdcb2-d7dd-486d-9522-27b5b8be64ad" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.27:443 10.132.0.12:55838 - v1-completions-path 2026-10-31T18:00:04.632136Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:00:30.501738Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T18:00:46.861Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 63 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "4539c212-690e-434e-9607-3057737540d7" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.27:34302 10.133.0.27:443 10.132.0.12:38742 - v1-completions-path [2026-10-31T18:00:47.210Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 58 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "613a98f1-9ecf-4b9c-bfc4-3f1a25d66726" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.27:50476 10.133.0.27:443 10.132.0.12:38746 - v1-completions-path [2026-10-31T18:00:47.444Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 53 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "adfb16e7-f9d9-4f5d-bea9-74cc9da17135" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.27:34302 10.133.0.27:443 10.132.0.12:38752 - v1-completions-path 2026-10-31T18:01:03.601726Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:01:13.863136Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:01:18.628063Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:01:35.378280Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:01:46.611900Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T18:01:51.856Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 74 291 99 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "0941495d-00a2-4754-93bc-b528fb8f908e" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "10.134.0.37:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.133.0.27:52308 10.133.0.27:443 10.132.0.12:45966 - v1-completions-path 2026-10-31T18:02:03.926052Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:02:08.492176Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:02:25.032096Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:02:41.254994Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:02:51.809052Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:02:54.841902Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:03:01.010048Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T18:03:01.010084Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T18:03:01.010098Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T18:03:01.010103Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-31T18:03:00.932Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 323 78 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "3feff7ce-b4e7-4bcd-8284-278305ad316c" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "10.132.0.54:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.27:36832 10.133.0.27:443 10.132.0.12:50676 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T18:03:13.907270Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:03:16.828241Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:03:39.240930Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:03:43.695421Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T18:03:43.695458Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T18:03:43.695474Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T18:03:43.695479Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-31T18:03:43.623Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 65 308 72 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "77b1424a-a996-43b5-83d0-8c1f18bd1a9d" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "10.132.0.54:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.27:36832 10.133.0.27:443 10.132.0.12:38050 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T18:03:56.118518Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:04:06.292399Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:04:36.262738Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:04:39.605937Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:06:23.783517Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:06:27.415215Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:07:44.998843Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T18:07:44.998880Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T18:07:44.998897Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T18:07:44.998902Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-31T18:07:44.911Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 313 87 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "9b8d10a6-c066-4ea6-b8da-5374b0dfdd62" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "10.132.0.54:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.27:46234 10.133.0.27:443 10.132.0.12:35588 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T18:08:35.650821Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:08:50.316176Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:08:52.809558Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:09:18.462483Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:11:08.527907Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:11:27.705263Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:11:51.097796Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:12:03.131203Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:12:09.582058Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:12:21.928814Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:12:25.690308Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:12:32.963852Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T18:12:32.963891Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T18:12:32.963909Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T18:12:32.963914Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-31T18:12:32.906Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 524 57 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "8e62ae28-44ec-4abd-b366-17fa8515e08c" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "10.132.0.54:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.27:49210 10.133.0.27:443 10.132.0.12:33858 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T18:12:45.481028Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:15:45.395358Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T18:15:45.395384Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T18:15:45.395387Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T18:15:45.395391Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-10-31T18:15:45.315Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 403 - - "-" 61 103 79 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "e7ab48da-52e0-4f47-ae7f-33468cfc36c1" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.27:443 10.132.0.12:46294 - v1-catch-all-path 2026-10-31T18:15:47.978217Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:15:53.177775Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T18:15:53.177815Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T18:15:53.177831Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T18:15:53.177836Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T18:15:53.281630Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-10-31T18:15:53.098Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 323 79 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "d740a59c-abd2-422c-b2ab-13c760545190" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "10.134.0.235:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.27:37998 10.133.0.27:443 10.132.0.12:59686 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T18:15:53.207Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 74 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "d596f4a6-93cc-94f4-b8fc-30f4cb240064" "maas.apps.61638e68a20cd568786d.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.27:55818 10.133.0.27:443 10.132.0.12:59688 - v1-catch-all-path 2026-10-31T18:15:56.092051Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:16:13.944898Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:16:16.343217Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:16:31.131457Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T18:16:41.983887Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8