2026-10-30T18:13:06.150146Z info FLAG: --concurrency="0" 2026-10-30T18:13:06.150164Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T18:13:06.150168Z info FLAG: --help="false" 2026-10-30T18:13:06.150171Z info FLAG: --log_as_json="false" 2026-10-30T18:13:06.150173Z info FLAG: --log_caller="" 2026-10-30T18:13:06.150175Z info FLAG: --log_output_level="default:info" 2026-10-30T18:13:06.150177Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T18:13:06.150182Z info FLAG: --log_target="[stdout]" 2026-10-30T18:13:06.150184Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T18:13:06.150186Z info FLAG: --outlierLogPath="" 2026-10-30T18:13:06.150189Z info FLAG: --profiling="true" 2026-10-30T18:13:06.150193Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T18:13:06.150200Z info FLAG: --proxyLogLevel="warning" 2026-10-30T18:13:06.150203Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T18:13:06.150206Z info FLAG: --stsPort="0" 2026-10-30T18:13:06.150209Z info FLAG: --templateFile="" 2026-10-30T18:13:06.150212Z info FLAG: --tokenManagerPlugin="" 2026-10-30T18:13:06.150221Z info FLAG: --vklog="0" 2026-10-30T18:13:06.150225Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T18:13:06.150389Z info Proxy role ips=[10.132.0.227] type=router id=maas-default-gateway-openshift-default-7df854bf67-pvlkq.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T18:13:06.150443Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T18:13:06.151578Z info cpu limit detected as 2, setting concurrency 2026-10-30T18:13:06.152258Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T18:13:06.152272Z info JWT policy is third-party-jwt 2026-10-30T18:13:06.152276Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T18:13:06.153609Z info Starting default Istio SDS Server 2026-10-30T18:13:06.153634Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T18:13:06.153658Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T18:13:06.154078Z info Opening status port 15020 2026-10-30T18:13:06.155423Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T18:13:06.155966Z info sds Starting SDS grpc server 2026-10-30T18:13:06.156007Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T18:13:06.156702Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T18:13:06.157598Z info Starting proxy agent 2026-10-30T18:13:06.157636Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T18:13:06.222702Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T18:13:06.249580Z info cache generated new workload certificate resourceName=default latency=93.474565ms ttl=23h59m59.750432205s 2026-10-30T18:13:06.249610Z info cache Root cert has changed, start rotating root cert 2026-10-30T18:13:06.249643Z info cache returned workload trust anchor from cache ttl=23h59m59.750357709s 2026-10-30T18:13:06.348230Z info ads ADS: new connection for node:1 2026-10-30T18:13:06.348299Z info cache returned workload certificate from cache ttl=23h59m59.651703036s 2026-10-30T18:13:06.348895Z info ads ADS: new connection for node:2 2026-10-30T18:13:06.348984Z info cache returned workload trust anchor from cache ttl=23h59m59.651017471s 2026-10-30T18:13:06.437539Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:13:12.777086Z error failed scraping envoy metrics: error scraping http://localhost:15090/stats/prometheus: Get "http://localhost:15090/stats/prometheus": dial tcp [::1]:15090: connect: connection refused 2026-10-30T18:13:14.373501Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:13:15.313661Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-10-30T18:13:15.743849Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-10-30T18:13:16.391145Z info Readiness succeeded in 10.24333141s 2026-10-30T18:13:16.391549Z info Envoy proxy is ready 2026-10-30T18:13:24.007771Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:13:41.533626Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:13:44.016010Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:13:47.147981Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:13:56.977960Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T18:14:13.613Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 80 - "52.71.36.72,10.132.0.21" "python-requests/2.34.2" "2b1aea9e-53c0-4ae5-b15e-cc189123006a" "maas.apps.43d793594a117fff08c9.prod.konflux-ocp-ci.dev" "10.133.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.227:59382 10.132.0.227:443 10.132.0.21:56964 - v1-completions-path [2026-10-30T18:14:14.060Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 70 - "52.71.36.72,10.132.0.21" "python-requests/2.34.2" "86bea23e-ec2e-4ad4-8174-7d61d3086750" "maas.apps.43d793594a117fff08c9.prod.konflux-ocp-ci.dev" "10.133.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.227:59384 10.132.0.227:443 10.132.0.21:56966 - v1-completions-path [2026-10-30T18:14:14.756Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 62 - "52.71.36.72,10.132.0.21" "python-requests/2.34.2" "3de685cf-c299-4277-bade-ee48b2329e47" "maas.apps.43d793594a117fff08c9.prod.konflux-ocp-ci.dev" "10.133.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.227:59382 10.132.0.227:443 10.132.0.21:56976 - v1-completions-path 2026-10-30T18:14:30.668559Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:14:41.093084Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:15:05.114339Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:15:22.702962Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:15:25.028101Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:15:44.313542Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:15:56.407545Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:16:14.023322Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:16:31.206604Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:16:41.903912Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:16:44.984168Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:17:04.277580Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:17:07.043755Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:17:15.221678Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T18:17:15.221715Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T18:17:15.221730Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T18:17:15.221735Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T18:17:15.142Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 78 - "52.71.36.72,10.132.0.21" "python-requests/2.34.2" "1269d52c-cbb6-4edd-a90e-ec1746f75dd7" "maas.apps.43d793594a117fff08c9.prod.konflux-ocp-ci.dev" "10.132.0.44:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.227:55498 10.132.0.227:443 10.132.0.21:35778 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T18:17:27.988532Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:17:30.722809Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:19:10.270965Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:19:13.748163Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:20:24.857509Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T18:20:24.857532Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T18:20:24.857536Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T18:20:24.857540Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-10-30T18:20:24.813Z] "POST /llm/e2e-trlp-test-simulated/v1/completions HTTP/1.1" 403 - - "-" 73 113 44 - "52.71.36.72,10.132.0.21" "python-requests/2.34.2" "18f8e89e-ca2f-4376-b1ed-bda0cbbf23c0" "maas.apps.43d793594a117fff08c9.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-trlp-test-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.227:443 10.132.0.21:38710 - v1-completions-path 2026-10-30T18:20:43.609990Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:20:59.199680Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:21:01.398382Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T18:21:02.056Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 129 - "52.71.36.72,10.132.0.21" "python-requests/2.34.2" "b67140de-1c82-4b45-9eb6-48aaa7561b92" "maas.apps.43d793594a117fff08c9.prod.konflux-ocp-ci.dev" "10.132.0.47:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.227:37612 10.132.0.227:443 10.132.0.21:51312 - v1-completions-path 2026-10-30T18:21:26.100622Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:23:23.909949Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:23:45.213567Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:24:11.597258Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:24:23.459497Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:24:27.369018Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T18:24:27.369057Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T18:24:27.369078Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T18:24:27.369085Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T18:24:27.284Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 84 326 84 - "52.71.36.72,10.132.0.21" "python-requests/2.34.2" "38793c5e-ca76-417b-85c3-6b4c1d58c167" "maas.apps.43d793594a117fff08c9.prod.konflux-ocp-ci.dev" "10.132.0.44:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.227:39300 10.132.0.227:443 10.132.0.21:50484 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T18:24:27.704236Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:24:39.387294Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:24:52.843036Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:25:29.981152Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:25:39.939197Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:25:48.334778Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:25:54.304223Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-10-30T18:25:54.296Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 429 - - "-" 61 18 56 - "52.71.36.72,10.132.0.21" "python-requests/2.34.2" "ade305d1-89b6-4214-9a5f-3770e3b7f77c" "maas.apps.43d793594a117fff08c9.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.227:443 10.132.0.21:35594 - v1-catch-all-path 2026-10-30T18:26:01.629898Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:26:03.907683Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:26:24.418845Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:26:35.301987Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8