2026-10-31T20:19:25.966701Z info FLAG: --concurrency="0" 2026-10-31T20:19:25.966719Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-31T20:19:25.966724Z info FLAG: --help="false" 2026-10-31T20:19:25.966727Z info FLAG: --log_as_json="false" 2026-10-31T20:19:25.966730Z info FLAG: --log_caller="" 2026-10-31T20:19:25.966733Z info FLAG: --log_output_level="default:info" 2026-10-31T20:19:25.966736Z info FLAG: --log_stacktrace_level="default:none" 2026-10-31T20:19:25.966742Z info FLAG: --log_target="[stdout]" 2026-10-31T20:19:25.966746Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-31T20:19:25.966749Z info FLAG: --outlierLogPath="" 2026-10-31T20:19:25.966752Z info FLAG: --profiling="true" 2026-10-31T20:19:25.966756Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-31T20:19:25.966759Z info FLAG: --proxyLogLevel="warning" 2026-10-31T20:19:25.966762Z info FLAG: --serviceCluster="istio-proxy" 2026-10-31T20:19:25.966765Z info FLAG: --stsPort="0" 2026-10-31T20:19:25.966768Z info FLAG: --templateFile="" 2026-10-31T20:19:25.966770Z info FLAG: --tokenManagerPlugin="" 2026-10-31T20:19:25.966774Z info FLAG: --vklog="0" 2026-10-31T20:19:25.966778Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-31T20:19:25.966937Z info Proxy role ips=[10.133.1.72] type=router id=maas-default-gateway-openshift-default-75d7c785f-68qbd.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-31T20:19:25.966991Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-31T20:19:25.968047Z info cpu limit detected as 2, setting concurrency 2026-10-31T20:19:25.968266Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-31T20:19:25.968278Z info JWT policy is third-party-jwt 2026-10-31T20:19:25.968282Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-31T20:19:25.970040Z info Starting default Istio SDS Server 2026-10-31T20:19:25.970065Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-31T20:19:25.970090Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-31T20:19:25.970271Z info Opening status port 15020 2026-10-31T20:19:25.971154Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-31T20:19:25.972995Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-31T20:19:25.973828Z info Starting proxy agent 2026-10-31T20:19:25.973862Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-31T20:19:25.983379Z info sds Starting SDS grpc server 2026-10-31T20:19:25.983404Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-31T20:19:26.041371Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-31T20:19:26.105920Z info cache generated new workload certificate resourceName=default latency=133.757305ms ttl=23h59m59.894082781s 2026-10-31T20:19:26.105950Z info cache Root cert has changed, start rotating root cert 2026-10-31T20:19:26.105985Z info cache returned workload trust anchor from cache ttl=23h59m59.894016549s 2026-10-31T20:19:26.147237Z info ads ADS: new connection for node:1 2026-10-31T20:19:26.147293Z info cache returned workload certificate from cache ttl=23h59m59.852708191s 2026-10-31T20:19:26.147536Z info ads ADS: new connection for node:2 2026-10-31T20:19:26.147602Z info cache returned workload trust anchor from cache ttl=23h59m59.852398986s 2026-10-31T20:19:26.233286Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T20:19:34.150341Z info Readiness succeeded in 8.186253187s 2026-10-31T20:19:34.150784Z info Envoy proxy is ready [2026-10-31T20:20:30.936Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 57 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "2fa08b6c-a6d6-4939-8c17-17d97d80d20f" "maas.apps.d4a8a5ec385124c63044.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.1.72:443 10.132.0.13:33396 - v1-completions-path 2026-10-31T20:20:32.566535Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T20:20:35.609188Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T20:22:02.794994Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T20:22:17.802864Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T20:22:19.422340Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T20:22:25.303980Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T20:22:25.304006Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T20:22:25.304009Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T20:22:25.304013Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-10-31T20:22:25.268Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 123 35 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "b978ef75-0eee-4520-a520-22a40d789b8a" "maas.apps.d4a8a5ec385124c63044.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.1.72:443 10.132.0.13:50822 - v1-completions-path 2026-10-31T20:22:36.789053Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T20:24:05.228322Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T20:24:20.530267Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T20:24:35.421564Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T20:24:35.421606Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T20:24:35.421628Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T20:24:35.421635Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T20:24:35.535532Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T20:24:35.535570Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T20:24:35.535587Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T20:24:35.535592Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-31T20:24:35.324Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 80 323 96 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "691529be-89c5-412d-af8d-67bda89aa285" "maas.apps.d4a8a5ec385124c63044.prod.konflux-ocp-ci.dev" "10.133.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.1.72:40488 10.133.1.72:443 10.132.0.13:39906 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T20:24:35.480Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 628 54 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "e944a6e4-bfbb-4a7c-9c72-f3205356e76a" "maas.apps.d4a8a5ec385124c63044.prod.konflux-ocp-ci.dev" "10.133.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.1.72:40488 10.133.1.72:443 10.132.0.13:39910 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T20:24:36.751909Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T20:24:48.473631Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T20:24:51.622519Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T20:25:03.318558Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T20:25:05.742642Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T20:25:11.507084Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T20:25:11.507120Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T20:25:11.507134Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T20:25:11.507140Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T20:25:11.890075Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-10-31T20:25:11.444Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 335 62 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "2095a018-0490-4fc1-a11b-82f03b4998c2" "maas.apps.d4a8a5ec385124c63044.prod.konflux-ocp-ci.dev" "10.133.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.1.72:40488 10.133.1.72:443 10.132.0.13:35904 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T20:25:11.700Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 75 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "af452f11-7330-4943-99aa-fcc054afbd6d" "maas.apps.d4a8a5ec385124c63044.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.72:52662 10.133.1.72:443 10.132.0.13:35910 - v1-completions-path [2026-10-31T20:25:11.832Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 59 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "2a4c7884-89c9-4154-9b9f-1738d78cb1a2" "maas.apps.d4a8a5ec385124c63044.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.72:52662 10.133.1.72:443 10.132.0.13:35914 - v1-completions-path [2026-10-31T20:25:11.951Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 62 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "67aaf47c-7cc8-4c0a-958b-740449b8badd" "maas.apps.d4a8a5ec385124c63044.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.1.72:443 10.132.0.13:35920 - v1-completions-path 2026-10-31T20:25:12.241274Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T20:25:12.241314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T20:25:12.241346Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T20:25:12.241352Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-31T20:25:12.190Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 524 50 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "70748bf0-d9ee-4a5e-b791-73b2bb472580" "maas.apps.d4a8a5ec385124c63044.prod.konflux-ocp-ci.dev" "10.133.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.1.72:40488 10.133.1.72:443 10.132.0.13:35926 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T20:25:23.807096Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T20:28:14.970023Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T20:28:20.275493Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T20:28:35.556253Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T20:28:37.151696Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T20:28:51.470865Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T20:29:02.301556Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9