2026-10-30T03:34:36.166878Z info FLAG: --concurrency="0" 2026-10-30T03:34:36.166896Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T03:34:36.166901Z info FLAG: --help="false" 2026-10-30T03:34:36.166903Z info FLAG: --log_as_json="false" 2026-10-30T03:34:36.166905Z info FLAG: --log_caller="" 2026-10-30T03:34:36.166907Z info FLAG: --log_output_level="default:info" 2026-10-30T03:34:36.166909Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T03:34:36.166913Z info FLAG: --log_target="[stdout]" 2026-10-30T03:34:36.166916Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T03:34:36.166917Z info FLAG: --outlierLogPath="" 2026-10-30T03:34:36.166919Z info FLAG: --profiling="true" 2026-10-30T03:34:36.166921Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T03:34:36.166923Z info FLAG: --proxyLogLevel="warning" 2026-10-30T03:34:36.166926Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T03:34:36.166928Z info FLAG: --stsPort="0" 2026-10-30T03:34:36.166930Z info FLAG: --templateFile="" 2026-10-30T03:34:36.166931Z info FLAG: --tokenManagerPlugin="" 2026-10-30T03:34:36.166934Z info FLAG: --vklog="0" 2026-10-30T03:34:36.166936Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T03:34:36.167084Z info Proxy role ips=[10.134.1.24] type=router id=maas-default-gateway-openshift-default-c9ffffc7f-8bc4f.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T03:34:36.167133Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T03:34:36.168257Z info cpu limit detected as 2, setting concurrency 2026-10-30T03:34:36.168481Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T03:34:36.168493Z info JWT policy is third-party-jwt 2026-10-30T03:34:36.168496Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T03:34:36.170076Z info Starting default Istio SDS Server 2026-10-30T03:34:36.170104Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T03:34:36.170147Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T03:34:36.170552Z info Opening status port 15020 2026-10-30T03:34:36.171325Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T03:34:36.172207Z info sds Starting SDS grpc server 2026-10-30T03:34:36.172228Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T03:34:36.172795Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T03:34:36.173607Z info Starting proxy agent 2026-10-30T03:34:36.173636Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T03:34:36.235743Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T03:34:36.303143Z info cache generated new workload certificate resourceName=default latency=130.745476ms ttl=23h59m59.696860652s 2026-10-30T03:34:36.303178Z info cache Root cert has changed, start rotating root cert 2026-10-30T03:34:36.303207Z info cache returned workload trust anchor from cache ttl=23h59m59.696793649s 2026-10-30T03:34:36.348750Z info ads ADS: new connection for node:1 2026-10-30T03:34:36.348811Z info cache returned workload certificate from cache ttl=23h59m59.651190645s 2026-10-30T03:34:36.349126Z info ads ADS: new connection for node:2 2026-10-30T03:34:36.349201Z info cache returned workload trust anchor from cache ttl=23h59m59.650800403s 2026-10-30T03:34:36.429625Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:34:42.309652Z error failed scraping envoy metrics: error scraping http://localhost:15090/stats/prometheus: Get "http://localhost:15090/stats/prometheus": dial tcp [::1]:15090: connect: connection refused 2026-10-30T03:34:44.110742Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:34:45.058569Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-10-30T03:34:45.781560Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-10-30T03:34:46.021844Z info Readiness succeeded in 9.857652059s 2026-10-30T03:34:46.022194Z info Envoy proxy is ready 2026-10-30T03:34:58.515110Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T03:35:04.234Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 298 97 - "34.228.250.194,10.132.0.8" "python-requests/2.34.2" "e5ae1fca-987d-4eaf-a3f5-fcd866c27b37" "maas.apps.bced66192e9659007ae2.prod.konflux-ocp-ci.dev" "10.133.0.44:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.24:44826 10.134.1.24:443 10.132.0.8:35372 - v1-completions-path [2026-10-30T03:35:05.312Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 66 - "34.228.250.194,10.132.0.8" "python-requests/2.34.2" "91864538-8504-4d0f-b119-f41c9d20da6d" "maas.apps.bced66192e9659007ae2.prod.konflux-ocp-ci.dev" "10.133.0.44:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.24:44838 10.134.1.24:443 10.132.0.8:35384 - v1-completions-path [2026-10-30T03:35:05.583Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 61 - "34.228.250.194,10.132.0.8" "python-requests/2.34.2" "b309ccaa-0988-4e62-ac4e-0dd08c862cd1" "maas.apps.bced66192e9659007ae2.prod.konflux-ocp-ci.dev" "10.133.0.44:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.24:44838 10.134.1.24:443 10.132.0.8:35392 - v1-completions-path [2026-10-30T03:35:07.063Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 66 - "34.228.250.194,10.132.0.8" "python-requests/2.34.2" "faa7b595-9f07-47cd-97a0-ca545779a4a8" "maas.apps.bced66192e9659007ae2.prod.konflux-ocp-ci.dev" "10.133.0.44:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.24:44826 10.134.1.24:443 10.132.0.8:35408 - v1-completions-path 2026-10-30T03:35:22.311300Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:35:32.668845Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:35:36.866759Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:35:41.433448Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-10-30T03:35:41.758875Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-10-30T03:35:41.374Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 60 - "34.228.250.194,10.132.0.8" "python-requests/2.34.2" "20ff4540-44cb-465b-bf7f-f27bc23ffea8" "maas.apps.bced66192e9659007ae2.prod.konflux-ocp-ci.dev" "10.133.0.44:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.24:44826 10.134.1.24:443 10.132.0.8:36566 - v1-completions-path [2026-10-30T03:35:41.688Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 70 - "34.228.250.194,10.132.0.8" "python-requests/2.34.2" "ed967a9d-157b-4ec1-ae8b-70174562da00" "maas.apps.bced66192e9659007ae2.prod.konflux-ocp-ci.dev" "10.133.0.44:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.24:44838 10.134.1.24:443 10.132.0.8:36580 - v1-completions-path [2026-10-30T03:35:42.223Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 53 - "34.228.250.194,10.132.0.8" "python-requests/2.34.2" "09295f39-f42f-4900-88e4-5c3eaf41015a" "maas.apps.bced66192e9659007ae2.prod.konflux-ocp-ci.dev" "10.133.0.44:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.24:44826 10.134.1.24:443 10.132.0.8:36584 - v1-completions-path 2026-10-30T03:35:54.226248Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:35:56.385265Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:36:15.829152Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:36:20.104924Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:36:36.957108Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:36:39.117693Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:36:58.268642Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:37:09.151838Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:37:11.904784Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:40:29.114734Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:40:29.114769Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:40:29.114788Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:40:29.114793Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T03:40:29.035Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 79 - "34.228.250.194,10.132.0.8" "python-requests/2.34.2" "142d2db7-7b88-41bf-82eb-38970e88dc76" "maas.apps.bced66192e9659007ae2.prod.konflux-ocp-ci.dev" "10.132.0.39:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.24:44736 10.134.1.24:443 10.132.0.8:44312 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T03:40:49.715084Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:41:05.106328Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:41:07.251743Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:41:21.835176Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:42:34.665104Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:42:34.665139Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:42:34.665152Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:42:34.665157Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T03:42:34.608Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 56 - "34.228.250.194,10.132.0.8" "python-requests/2.34.2" "0432810b-db21-414b-9885-99312ae14bea" "maas.apps.bced66192e9659007ae2.prod.konflux-ocp-ci.dev" "10.132.0.39:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.24:60722 10.134.1.24:443 10.132.0.8:37148 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T03:42:41.591664Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:42:56.093326Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:43:11.644501Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:43:26.132497Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:43:29.484241Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:43:29.484279Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:43:29.484294Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:43:29.484299Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T03:43:29.894015Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T03:43:29.404Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 84 326 79 - "34.228.250.194,10.132.0.8" "python-requests/2.34.2" "ae88eaa6-bf0a-43a3-a0bf-36685bf3e1b6" "maas.apps.bced66192e9659007ae2.prod.konflux-ocp-ci.dev" "10.132.0.39:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.24:60722 10.134.1.24:443 10.132.0.8:58144 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T03:43:41.518692Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:43:46.210968Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:44:04.115616Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:44:14.099133Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:44:22.402630Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:44:28.216681Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-10-30T03:44:28.142Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 74 - "34.228.250.194,10.132.0.8" "python-requests/2.34.2" "e89e7403-8b99-4107-85d6-ef7945358adc" "maas.apps.bced66192e9659007ae2.prod.konflux-ocp-ci.dev" "10.133.0.43:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.1.24:54444 10.134.1.24:443 10.132.0.8:36684 - v1-catch-all-path [2026-10-30T03:44:28.518Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 429 - - "-" 61 18 55 - "34.228.250.194,10.132.0.8" "python-requests/2.34.2" "4f9dada6-5744-4685-9b7b-7154393319cd" "maas.apps.bced66192e9659007ae2.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.24:443 10.132.0.8:36694 - v1-catch-all-path 2026-10-30T03:44:35.825991Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:44:38.088572Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:44:52.655499Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:45:03.779681Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8