2026-11-02T07:03:31.801559Z info FLAG: --concurrency="0" 2026-11-02T07:03:31.801577Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-02T07:03:31.801581Z info FLAG: --help="false" 2026-11-02T07:03:31.801584Z info FLAG: --log_as_json="false" 2026-11-02T07:03:31.801586Z info FLAG: --log_caller="" 2026-11-02T07:03:31.801588Z info FLAG: --log_output_level="default:info" 2026-11-02T07:03:31.801590Z info FLAG: --log_stacktrace_level="default:none" 2026-11-02T07:03:31.801594Z info FLAG: --log_target="[stdout]" 2026-11-02T07:03:31.801596Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-02T07:03:31.801598Z info FLAG: --outlierLogPath="" 2026-11-02T07:03:31.801600Z info FLAG: --profiling="true" 2026-11-02T07:03:31.801603Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-02T07:03:31.801605Z info FLAG: --proxyLogLevel="warning" 2026-11-02T07:03:31.801606Z info FLAG: --serviceCluster="istio-proxy" 2026-11-02T07:03:31.801609Z info FLAG: --stsPort="0" 2026-11-02T07:03:31.801610Z info FLAG: --templateFile="" 2026-11-02T07:03:31.801612Z info FLAG: --tokenManagerPlugin="" 2026-11-02T07:03:31.801614Z info FLAG: --vklog="0" 2026-11-02T07:03:31.801617Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-02T07:03:31.801753Z info Proxy role ips=[10.133.0.145] type=router id=maas-default-gateway-openshift-default-55f85cf5d7-8xsqm.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-02T07:03:31.801809Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-02T07:03:31.802855Z info cpu limit detected as 2, setting concurrency 2026-11-02T07:03:31.803047Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-02T07:03:31.803058Z info JWT policy is third-party-jwt 2026-11-02T07:03:31.803061Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-02T07:03:31.804883Z info Starting default Istio SDS Server 2026-11-02T07:03:31.804910Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-02T07:03:31.804927Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-02T07:03:31.805359Z info Opening status port 15020 2026-11-02T07:03:31.806377Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-02T07:03:31.807629Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-02T07:03:31.808500Z info Starting proxy agent 2026-11-02T07:03:31.808530Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-02T07:03:31.818250Z info sds Starting SDS grpc server 2026-11-02T07:03:31.818277Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-02T07:03:31.877141Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-02T07:03:32.030008Z info ads ADS: new connection for node:1 2026-11-02T07:03:32.030734Z info ads ADS: new connection for node:2 2026-11-02T07:03:32.201396Z info cache generated new workload certificate resourceName=default latency=395.1521ms ttl=23h59m59.798607994s 2026-11-02T07:03:32.201433Z info cache Root cert has changed, start rotating root cert 2026-11-02T07:03:32.201463Z info cache returned workload certificate from cache ttl=23h59m59.798539622s 2026-11-02T07:03:32.201476Z info cache returned workload trust anchor from cache ttl=23h59m59.798524405s 2026-11-02T07:03:32.201559Z info cache returned workload trust anchor from cache ttl=23h59m59.798441415s 2026-11-02T07:03:32.201943Z info cache returned workload trust anchor from cache ttl=23h59m59.798058056s 2026-11-02T07:03:32.303817Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:03:37.956029Z error failed scraping envoy metrics: error scraping http://localhost:15090/stats/prometheus: Get "http://localhost:15090/stats/prometheus": dial tcp [::1]:15090: connect: connection refused 2026-11-02T07:03:40.310897Z info Readiness succeeded in 8.512012061s 2026-11-02T07:03:40.311371Z info Envoy proxy is ready 2026-11-02T07:03:45.037035Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:04:32.611993Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-02T07:04:32.612018Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-02T07:04:32.612022Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-02T07:04:32.612027Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-11-02T07:04:32.565Z] "POST /llm/premium-simulated-simulated-premium/v1/completions HTTP/1.1" 403 - - "-" 74 77 46 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "e389d75c-b218-4934-ab07-c4c719d62b17" "maas.apps.d46096a673f3dec2b42d.prod.konflux-ocp-ci.dev" "-" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.145:443 10.132.0.17:46568 - v1-completions-path 2026-11-02T07:04:39.621692Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-11-02T07:04:45.472Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 76 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "2dfab1fe-3f40-429f-9fc5-3584758feee5" "maas.apps.d46096a673f3dec2b42d.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.145:47578 10.133.0.145:443 10.132.0.17:33024 - v1-completions-path 2026-11-02T07:05:01.031218Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:05:05.532249Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-11-02T07:05:17.810Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 54 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "464dd454-1dd2-4041-bc68-157e21d7212c" "maas.apps.d46096a673f3dec2b42d.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.145:443 10.132.0.17:46084 - v1-completions-path 2026-11-02T07:05:35.251087Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:05:48.528252Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-11-02T07:05:56.388Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 61 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "40b145fc-4a91-477e-a63c-a01a513c1e52" "maas.apps.d46096a673f3dec2b42d.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.145:47578 10.133.0.145:443 10.132.0.17:46930 - v1-completions-path 2026-11-02T07:05:57.567293Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-11-02T07:05:57.515Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 52 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "dbf5c93b-6421-4e81-8e4f-4949a87d7cf0" "maas.apps.d46096a673f3dec2b42d.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.145:443 10.132.0.17:59990 - v1-completions-path [2026-11-02T07:05:57.764Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 59 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "75fad9f3-66a6-4539-907d-9d03d740ee26" "maas.apps.d46096a673f3dec2b42d.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.145:40624 10.133.0.145:443 10.132.0.17:59992 - v1-completions-path [2026-11-02T07:06:02.536Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 300 63 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "d0e23f0d-5265-4ddb-a2a3-eb03ac5125b4" "maas.apps.d46096a673f3dec2b42d.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.145:47578 10.133.0.145:443 10.132.0.17:60012 - v1-completions-path 2026-11-02T07:06:19.520680Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:06:30.112540Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:06:36.230168Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-11-02T07:06:48.689Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 61 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "241200e5-f2f0-42ec-994a-729d42f02c82" "maas.apps.d46096a673f3dec2b42d.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.145:47578 10.133.0.145:443 10.132.0.17:56342 - v1-completions-path 2026-11-02T07:07:05.029619Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:07:09.580891Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:07:32.877136Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:07:39.025450Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:08:05.243015Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:08:24.254656Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:08:37.210808Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:08:42.893366Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:09:07.852650Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:09:13.227170Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-11-02T07:09:22.836Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 278 70 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "5195a0b6-d1a9-49a6-aae1-179afa4f7b56" "maas.apps.d46096a673f3dec2b42d.prod.konflux-ocp-ci.dev" "10.133.0.30:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.145:53534 10.133.0.145:443 10.132.0.17:47868 - v1-completions-path 2026-11-02T07:09:31.731529Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:09:38.795369Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T07:09:38.795405Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T07:09:38.795420Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T07:09:38.795425Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T07:09:38.684Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 67 310 110 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "ad492734-5c47-42c4-b7ba-a7a7b9dc7f6f" "maas.apps.d46096a673f3dec2b42d.prod.konflux-ocp-ci.dev" "10.133.0.23:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.145:56088 10.133.0.145:443 10.132.0.17:51654 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T07:09:53.443021Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:10:03.921196Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:10:22.753869Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:10:28.268111Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:12:53.666596Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:12:59.424278Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:14:16.561037Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-02T07:14:16.561061Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-02T07:14:16.561063Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-02T07:14:16.561068Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-11-02T07:14:16.519Z] "POST /llm/e2e-trlp-test-simulated/v1/completions HTTP/1.1" 403 - - "-" 73 113 41 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "9522995e-2dc2-410c-9102-cfca56ca8685" "maas.apps.d46096a673f3dec2b42d.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-trlp-test-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.145:443 10.132.0.17:44234 - v1-completions-path 2026-11-02T07:14:38.244257Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:14:55.421852Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:15:00.162449Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:15:24.364210Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:17:28.531868Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:18:08.417461Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:18:36.719198Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:18:51.905240Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:19:04.898122Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:19:17.044968Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:19:21.874663Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:19:48.786386Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:23:18.028225Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:23:35.308714Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:23:56.222513Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:24:00.958194Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:24:15.557484Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T07:24:26.909465Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7