2026-10-30T20:48:50.919556Z info FLAG: --concurrency="0" 2026-10-30T20:48:50.919581Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T20:48:50.919585Z info FLAG: --help="false" 2026-10-30T20:48:50.919588Z info FLAG: --log_as_json="false" 2026-10-30T20:48:50.919590Z info FLAG: --log_caller="" 2026-10-30T20:48:50.919592Z info FLAG: --log_output_level="default:info" 2026-10-30T20:48:50.919594Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T20:48:50.919598Z info FLAG: --log_target="[stdout]" 2026-10-30T20:48:50.919600Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T20:48:50.919602Z info FLAG: --outlierLogPath="" 2026-10-30T20:48:50.919604Z info FLAG: --profiling="true" 2026-10-30T20:48:50.919606Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T20:48:50.919608Z info FLAG: --proxyLogLevel="warning" 2026-10-30T20:48:50.919611Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T20:48:50.919613Z info FLAG: --stsPort="0" 2026-10-30T20:48:50.919615Z info FLAG: --templateFile="" 2026-10-30T20:48:50.919616Z info FLAG: --tokenManagerPlugin="" 2026-10-30T20:48:50.919619Z info FLAG: --vklog="0" 2026-10-30T20:48:50.919622Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T20:48:50.919807Z info Proxy role ips=[10.134.0.31] type=router id=maas-default-gateway-openshift-default-6db586dd6c-spcjv.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T20:48:50.919860Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T20:48:50.921060Z info cpu limit detected as 2, setting concurrency 2026-10-30T20:48:50.921267Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T20:48:50.921278Z info JWT policy is third-party-jwt 2026-10-30T20:48:50.921281Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T20:48:50.922715Z info Starting default Istio SDS Server 2026-10-30T20:48:50.922737Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T20:48:50.922753Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T20:48:50.923735Z info Opening status port 15020 2026-10-30T20:48:50.924294Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T20:48:50.925647Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T20:48:50.926505Z info Starting proxy agent 2026-10-30T20:48:50.926539Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T20:48:50.944492Z info sds Starting SDS grpc server 2026-10-30T20:48:50.944521Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T20:48:50.990617Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T20:48:51.021122Z info cache generated new workload certificate resourceName=default latency=97.104565ms ttl=23h59m59.978883445s 2026-10-30T20:48:51.021167Z info cache Root cert has changed, start rotating root cert 2026-10-30T20:48:51.021207Z info cache returned workload trust anchor from cache ttl=23h59m59.978794164s 2026-10-30T20:48:51.090906Z info ads ADS: new connection for node:1 2026-10-30T20:48:51.091009Z info cache returned workload certificate from cache ttl=23h59m59.908994432s 2026-10-30T20:48:51.091448Z info ads ADS: new connection for node:2 2026-10-30T20:48:51.091541Z info cache returned workload trust anchor from cache ttl=23h59m59.908460869s 2026-10-30T20:48:51.174716Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:48:59.096195Z info Readiness succeeded in 8.179675262s 2026-10-30T20:48:59.096532Z info Envoy proxy is ready 2026-10-30T20:49:29.059844Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:49:29.059884Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:49:29.059898Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T20:49:29.059903Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T20:49:28.977Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 320 81 - "34.228.250.194,10.132.0.10" "curl/7.76.1" "0d1a06ee-fbcb-4e78-a031-af200cf83d7f" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.58:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.31:47738 10.134.0.31:443 10.132.0.10:42736 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T20:49:29.467Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 406 74 - "34.228.250.194,10.132.0.10" "curl/7.76.1" "2edd93b0-62b7-491d-8514-9e0586191c6a" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.133.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.31:35094 10.134.0.31:443 10.132.0.10:42744 - v1-model-routing [2026-10-30T20:49:29.980Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 54 - "34.228.250.194,10.132.0.10" "curl/7.76.1" "d12206ae-1e8b-4ac4-80e8-520fcc6a2892" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.31:443 10.132.0.10:42758 - v1-model-routing 2026-10-30T20:49:30.743635Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T20:49:30.743684Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T20:49:30.743688Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T20:49:30.743691Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-10-30T20:49:30.217Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 53 - "34.228.250.194,10.132.0.10" "curl/7.76.1" "1595c0c7-23b4-46a9-89b0-a2a5f81b8ec2" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.31:443 10.132.0.10:42774 - v1-model-routing [2026-10-30T20:49:30.725Z] "POST /v1/chat/completions HTTP/1.1" 401 - - "-" 122 88 17 - "34.228.250.194,10.132.0.10" "curl/7.76.1" "697ed823-b652-40df-9be7-11948bcdbeae" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.31:443 10.132.0.10:42788 - v1-model-routing 2026-10-30T20:51:44.725892Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:51:44.725939Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:51:44.725960Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T20:51:44.725967Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T20:51:44.655Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 70 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "0ba0dc8e-73a9-4cf4-9fa0-73c48c69d330" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.58:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.31:42488 10.134.0.31:443 10.132.0.10:43868 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T20:51:44.915Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 42 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "cafa0cc6-98cf-4c10-8d68-989de7c61a17" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.134.0.31:443 10.132.0.10:43870 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T20:51:46.745157Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:51:46.745319Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:51:46.745354Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:51:46.745363Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T20:51:46.893448Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:51:46.893552Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:51:46.893579Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:51:46.893586Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T20:51:47.012344Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:51:47.012383Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:51:47.012398Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:51:47.012403Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T20:51:46.584Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 39 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "d874d725-c068-4792-a8d3-d4b69dd0a8a7" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.134.0.31:443 10.132.0.10:41376 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T20:51:46.681Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 64 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "c00cb546-e165-4264-b082-fe278771a5ee" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.58:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.31:42488 10.134.0.31:443 10.132.0.10:41392 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T20:51:46.824Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 69 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "397e5ea3-0a06-4d35-8636-71e362d08637" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.58:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.31:42490 10.134.0.31:443 10.132.0.10:41398 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T20:51:46.946Z] "DELETE /v1/api-keys/56c3842d-44a4-49f7-ad89-cfc8c25bf399 HTTP/1.1" 200 - via_upstream - "-" 0 321 65 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "ab906214-c8db-4770-a317-7b93b9e6b17f" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.58:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.31:42488 10.134.0.31:443 10.132.0.10:41412 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T20:51:47.066Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 46 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "4412c145-4d7f-4009-8066-6a75d7da1018" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.134.0.31:443 10.132.0.10:41418 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T20:51:47.268590Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:51:47.268628Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:51:47.268642Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T20:51:47.268647Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T20:51:47.169Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 99 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "1e75cb3a-d979-41e0-9482-1245176464a4" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.58:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.31:42488 10.134.0.31:443 10.132.0.10:41432 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T20:51:52.792883Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-10-30T20:51:52.868065Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:51:52.868225Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:51:52.868281Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:51:52.868312Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T20:51:52.770Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 98 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "6f720a38-eb65-4772-b4fa-3bde6d79631e" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.58:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.31:42488 10.134.0.31:443 10.132.0.10:41452 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T20:51:54.513338Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-10-30T20:51:54.452Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 288 65 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "bcfce0c0-400c-4390-843a-3bae42f56217" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.133.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.31:35094 10.134.0.31:443 10.132.0.10:41472 - v1-completions-path [2026-10-30T20:51:54.575Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 61 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "6ad902f3-980a-407d-a2ed-0386ab89802a" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.133.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.31:35094 10.134.0.31:443 10.132.0.10:41474 - v1-completions-path 2026-10-30T20:51:55.482198Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:51:55.482246Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:51:55.482269Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T20:51:55.482275Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T20:51:55.427Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 55 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "fa57859c-cfa6-4c61-9725-1d87236946df" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.58:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.31:42488 10.134.0.31:443 10.132.0.10:38452 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T20:51:55.864Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 68 - "34.228.250.194,10.132.0.10" "-" "0f9d2c70-7c56-4f35-985f-93d31d049df2" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.133.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.31:41180 10.134.0.31:443 10.132.0.10:38460 - v1-completions-path 2026-10-30T20:52:07.051069Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:52:21.882057Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T20:56:10.115Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 136 347 67 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "d37f442c-2283-4e38-b777-57ca252e1bb0" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.133.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.31:35094 10.134.0.31:443 10.132.0.10:60276 - v1-chat-completions-path 2026-10-30T20:56:14.834330Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:56:26.650521Z warn xdsproxy sendDownstreamDelta took 11.444022905s 2026-10-30T20:56:45.201166Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:57:23.914629Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:58:28.457594Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:03:01.749310Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T21:03:01.749374Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T21:03:01.749390Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T21:03:01.749396Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T21:03:01.646Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 30 315 102 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "3b3d94db-8173-4068-bc32-4bf42ac51dac" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.58:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.31:39428 10.134.0.31:443 10.132.0.10:47534 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T21:03:04.427278Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:03:26.652050Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:04:17.208620Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T21:04:23.128Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 76 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "fbf0adbf-2ad1-4c14-8001-9efa7f6b91a0" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.63:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.31:56282 10.134.0.31:443 10.132.0.10:52432 - v1-completions-path 2026-10-30T21:04:35.572197Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:04:37.473342Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:04:42.735957Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T21:04:42.735994Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T21:04:42.736008Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T21:04:42.736012Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T21:04:42.678Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 170 57 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "52229189-ae3f-474d-8fc9-bd3f642e7763" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.63:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.31:56282 10.134.0.31:443 10.132.0.10:45274 - v1-catch-all-path 2026-10-30T21:04:54.243432Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:04:56.140716Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:04:58.967681Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:05:06.556618Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:05:16.878912Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:05:19.816864Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-10-30T21:05:19.826243Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T21:05:19.826282Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T21:05:19.826298Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T21:05:19.826303Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T21:05:19.710Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 322 116 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "830cbb2c-1a65-48a0-9913-82471cebfaf5" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.58:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.31:48934 10.134.0.31:443 10.132.0.10:59532 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T21:05:20.997Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 60 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "53bbc6f1-26a0-41fa-bfcf-0d5692ffd9ac" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.63:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.31:56282 10.134.0.31:443 10.132.0.10:59558 - v1-completions-path 2026-10-30T21:05:38.829530Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:05:48.937311Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:05:50.724084Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:06:09.876584Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:06:11.635569Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:06:28.171083Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:06:30.451721Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:06:46.938696Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:06:48.790529Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:07:14.766158Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:07:25.469183Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:07:28.162316Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:07:35.091257Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T21:07:35.091303Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T21:07:35.091320Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T21:07:35.091326Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T21:07:35.022Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 324 69 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "3c30cdab-d63f-4207-997c-093d63699073" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.58:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.31:37482 10.134.0.31:443 10.132.0.10:38036 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T21:07:47.467239Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:07:49.709039Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:08:07.384900Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:08:25.874923Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:08:36.083857Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:09:02.611076Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:09:05.526780Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:10:40.471008Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:10:43.515597Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:12:20.760902Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:12:34.959409Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:12:46.220213Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:13:01.013226Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:15:03.817142Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T21:15:03.817180Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T21:15:03.817198Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T21:15:03.817203Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T21:15:03.749Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 67 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "71aff9a2-8c7b-4b93-9398-c7f5fc71971f" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "10.132.0.58:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.31:45228 10.134.0.31:443 10.132.0.10:34724 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T21:15:07.274077Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:15:24.832483Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:15:39.793144Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:16:01.730363Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:16:05.614448Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:16:17.715310Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:16:28.842081Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:16:46.048781Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:19:20.300573Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-10-30T21:19:48.331235Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T21:19:48.331269Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T21:19:48.331272Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T21:19:48.331277Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-10-30T21:19:48.275Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 403 - - "-" 61 103 56 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "9603ca57-f36b-4c39-8657-6a6671bfcaed" "maas.apps.a24b3162596f31397072.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.31:443 10.132.0.10:32860 - v1-catch-all-path 2026-10-30T21:19:50.226909Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:19:55.434004Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:20:10.584161Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:20:12.287668Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:20:26.641012Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:20:37.345442Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8