2026-10-31T14:05:14.517647Z info FLAG: --concurrency="0" 2026-10-31T14:05:14.517670Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-31T14:05:14.517675Z info FLAG: --help="false" 2026-10-31T14:05:14.517679Z info FLAG: --log_as_json="false" 2026-10-31T14:05:14.517682Z info FLAG: --log_caller="" 2026-10-31T14:05:14.517685Z info FLAG: --log_output_level="default:info" 2026-10-31T14:05:14.517688Z info FLAG: --log_stacktrace_level="default:none" 2026-10-31T14:05:14.517695Z info FLAG: --log_target="[stdout]" 2026-10-31T14:05:14.517698Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-31T14:05:14.517701Z info FLAG: --outlierLogPath="" 2026-10-31T14:05:14.517704Z info FLAG: --profiling="true" 2026-10-31T14:05:14.517707Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-31T14:05:14.517711Z info FLAG: --proxyLogLevel="warning" 2026-10-31T14:05:14.517714Z info FLAG: --serviceCluster="istio-proxy" 2026-10-31T14:05:14.517717Z info FLAG: --stsPort="0" 2026-10-31T14:05:14.517720Z info FLAG: --templateFile="" 2026-10-31T14:05:14.517723Z info FLAG: --tokenManagerPlugin="" 2026-10-31T14:05:14.517733Z info FLAG: --vklog="0" 2026-10-31T14:05:14.517737Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-31T14:05:14.517919Z info Proxy role ips=[10.132.1.78] type=router id=maas-default-gateway-openshift-default-5d84bc6b95-tvm4p.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-31T14:05:14.517976Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-31T14:05:14.519365Z info cpu limit detected as 2, setting concurrency 2026-10-31T14:05:14.519559Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-31T14:05:14.519570Z info JWT policy is third-party-jwt 2026-10-31T14:05:14.519573Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-31T14:05:14.521745Z info Starting default Istio SDS Server 2026-10-31T14:05:14.522098Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-31T14:05:14.522097Z info Opening status port 15020 2026-10-31T14:05:14.522134Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-31T14:05:14.522688Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-31T14:05:14.523231Z info sds Starting SDS grpc server 2026-10-31T14:05:14.523245Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-31T14:05:14.524191Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-31T14:05:14.524978Z info Starting proxy agent 2026-10-31T14:05:14.525009Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-31T14:05:14.601249Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-31T14:05:14.610329Z info cache generated new workload certificate resourceName=default latency=86.928409ms ttl=23h59m59.389674781s 2026-10-31T14:05:14.610426Z info cache Root cert has changed, start rotating root cert 2026-10-31T14:05:14.610490Z info cache returned workload trust anchor from cache ttl=23h59m59.389510731s 2026-10-31T14:05:14.712650Z info ads ADS: new connection for node:1 2026-10-31T14:05:14.712683Z info ads ADS: new connection for node:2 2026-10-31T14:05:14.712730Z info cache returned workload trust anchor from cache ttl=23h59m59.287271501s 2026-10-31T14:05:14.712732Z info cache returned workload certificate from cache ttl=23h59m59.28726954s 2026-10-31T14:05:14.805358Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:05:22.923339Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:05:23.858756Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-10-31T14:05:24.139345Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-10-31T14:05:25.320902Z info Readiness succeeded in 10.806501001s 2026-10-31T14:05:25.321252Z info Readiness succeeded in 10.80685659s 2026-10-31T14:05:25.321465Z info Envoy proxy is ready 2026-10-31T14:05:58.487566Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T14:05:58.487641Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T14:05:58.487658Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T14:05:58.487663Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-31T14:05:58.406Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 74 316 81 - "34.228.250.194,10.133.0.12" "python-requests/2.34.2" "ce2fb768-01f2-469b-a5c3-5b39cf5d34c8" "maas.apps.eea73e210ab98c75620c.prod.konflux-ocp-ci.dev" "10.133.0.54:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.78:55406 10.132.1.78:443 10.133.0.12:38372 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T14:06:01.440042Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:06:17.233217Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:06:19.192034Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:06:38.301629Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:06:40.008698Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:06:42.960793Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:06:51.245715Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:07:15.715740Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:07:25.931849Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:07:27.754386Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T14:07:34.452Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 85 - "34.228.250.194,10.133.0.12" "python-requests/2.34.2" "6803bd6f-8fb0-4d16-9b67-924046d790b6" "maas.apps.eea73e210ab98c75620c.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.78:34912 10.132.1.78:443 10.133.0.12:51978 - v1-completions-path 2026-10-31T14:07:46.840084Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:07:48.637890Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:07:55.250227Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-10-31T14:08:05.094504Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:08:07.490996Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:08:23.938334Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:08:25.651453Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:08:41.930679Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:08:52.543334Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:08:55.319962Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:09:21.091787Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T14:09:27.535Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 64 - "34.228.250.194,10.133.0.12" "python-requests/2.34.2" "b546a9b0-615e-43ea-9e72-9298fdabd8f2" "maas.apps.eea73e210ab98c75620c.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.78:44972 10.132.1.78:443 10.133.0.12:37044 - v1-completions-path 2026-10-31T14:09:39.432179Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:09:49.715234Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T14:10:04.383Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 38 - "34.228.250.194,10.133.0.12" "python-requests/2.34.2" "0538f071-71ca-4df7-bafc-747633441ad8" "maas.apps.eea73e210ab98c75620c.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.1.78:443 10.133.0.12:33294 - v1-completions-path 2026-10-31T14:10:06.071234Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:10:08.884091Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T14:11:47.987Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 50 - "34.228.250.194,10.133.0.12" "python-requests/2.34.2" "8c40f98d-9f1d-470e-9a8d-3bac79f3841a" "maas.apps.eea73e210ab98c75620c.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.1.78:443 10.133.0.12:50858 - v1-completions-path 2026-10-31T14:11:49.826216Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:11:52.670517Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:13:00.423719Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T14:13:00.423746Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T14:13:00.423749Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T14:13:00.423753Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-10-31T14:13:00.377Z] "POST /llm/e2e-trlp-test-simulated/v1/completions HTTP/1.1" 403 - - "-" 73 113 46 - "34.228.250.194,10.133.0.12" "python-requests/2.34.2" "f93f2667-698b-4768-bbce-2848020ef1af" "maas.apps.eea73e210ab98c75620c.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-trlp-test-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.1.78:443 10.133.0.12:54194 - v1-completions-path 2026-10-31T14:13:25.300792Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T14:13:27.752Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 73 292 127 - "34.228.250.194,10.133.0.12" "python-requests/2.34.2" "63dfa394-de20-4976-b571-8dbcc825fa58" "maas.apps.eea73e210ab98c75620c.prod.konflux-ocp-ci.dev" "10.134.0.32:8000" outbound|8000||e2e-trlp-test-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.1.78:49520 10.132.1.78:443 10.133.0.12:42390 - v1-completions-path 2026-10-31T14:13:40.396104Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:13:42.283307Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:14:01.491962Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:15:30.241158Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:15:46.765647Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:16:01.764906Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:16:13.514467Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:16:17.184385Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:16:28.084679Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:16:30.128118Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:16:49.687967Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:19:40.682253Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:19:43.816749Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-10-31T14:19:43.728Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 88 - "34.228.250.194,10.133.0.12" "python-requests/2.34.2" "e8e57b28-5ca3-40b9-88a8-d4c1d35cc8e0" "maas.apps.eea73e210ab98c75620c.prod.konflux-ocp-ci.dev" "10.132.0.40:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.1.78:58448 10.132.1.78:443 10.133.0.12:37838 - v1-catch-all-path 2026-10-31T14:19:45.870980Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:20:01.021770Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:20:02.722077Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:20:16.990704Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T14:20:27.909264Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8