2026-11-03T20:18:28.620602Z info FLAG: --concurrency="0" 2026-11-03T20:18:28.620622Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T20:18:28.620626Z info FLAG: --help="false" 2026-11-03T20:18:28.620629Z info FLAG: --log_as_json="false" 2026-11-03T20:18:28.620631Z info FLAG: --log_caller="" 2026-11-03T20:18:28.620633Z info FLAG: --log_output_level="default:info" 2026-11-03T20:18:28.620635Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T20:18:28.620640Z info FLAG: --log_target="[stdout]" 2026-11-03T20:18:28.620642Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T20:18:28.620644Z info FLAG: --outlierLogPath="" 2026-11-03T20:18:28.620646Z info FLAG: --profiling="true" 2026-11-03T20:18:28.620648Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T20:18:28.620650Z info FLAG: --proxyLogLevel="warning" 2026-11-03T20:18:28.620652Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T20:18:28.620654Z info FLAG: --stsPort="0" 2026-11-03T20:18:28.620656Z info FLAG: --templateFile="" 2026-11-03T20:18:28.620658Z info FLAG: --tokenManagerPlugin="" 2026-11-03T20:18:28.620661Z info FLAG: --vklog="0" 2026-11-03T20:18:28.620663Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T20:18:28.620873Z info Proxy role ips=[10.134.0.70] type=router id=maas-default-gateway-openshift-default-694db55c87-657vf.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T20:18:28.620926Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T20:18:28.622429Z info cpu limit detected as 2, setting concurrency 2026-11-03T20:18:28.622636Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T20:18:28.622648Z info JWT policy is third-party-jwt 2026-11-03T20:18:28.622651Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T20:18:28.625360Z info Starting default Istio SDS Server 2026-11-03T20:18:28.625388Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T20:18:28.625421Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T20:18:28.625591Z info Opening status port 15020 2026-11-03T20:18:28.626333Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T20:18:28.627715Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T20:18:28.628522Z info Starting proxy agent 2026-11-03T20:18:28.628559Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T20:18:28.648130Z info sds Starting SDS grpc server 2026-11-03T20:18:28.648170Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T20:18:28.715321Z info cache generated new workload certificate resourceName=default latency=88.900371ms ttl=23h59m59.284684444s 2026-11-03T20:18:28.715456Z info cache Root cert has changed, start rotating root cert 2026-11-03T20:18:28.715528Z info cache returned workload trust anchor from cache ttl=23h59m59.284473789s 2026-11-03T20:18:28.740976Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T20:18:28.929179Z info ads ADS: new connection for node:1 2026-11-03T20:18:28.929255Z info cache returned workload certificate from cache ttl=23h59m59.070746391s 2026-11-03T20:18:28.929496Z info ads ADS: new connection for node:2 2026-11-03T20:18:28.929567Z info cache returned workload trust anchor from cache ttl=23h59m59.07043376s 2026-11-03T20:18:29.046409Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:18:37.685540Z info Readiness succeeded in 9.067571893s 2026-11-03T20:18:37.686040Z info Envoy proxy is ready 2026-11-03T20:21:22.305982Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T20:21:22.306017Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T20:21:22.306030Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T20:21:22.306035Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-03T20:21:22.210Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 321 95 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "1966c400-dad5-46e8-863f-3a1654e35156" "maas.apps.e7be3e6863d7d8b1aa59.prod.konflux-ocp-ci.dev" "10.132.0.50:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.70:37826 10.134.0.70:443 10.132.0.14:46186 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T20:22:18.738403Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:22:55.124053Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:24:56.549978Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:25:34.034200Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:28:51.923110Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:29:34.421320Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:30:51.374488Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:31:33.984174Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:32:08.922051Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:39:00.861834Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:39:23.809067Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:40:13.720280Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:40:30.630218Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:40:35.393349Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:40:53.560654Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:41:09.244075Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:41:32.516388Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:41:42.842898Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:41:47.360821Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T20:41:52.015Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 80 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "d7c8d8fd-a244-461e-8102-e2411f1ceb97" "maas.apps.e7be3e6863d7d8b1aa59.prod.konflux-ocp-ci.dev" "10.134.0.32:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.70:40552 10.134.0.70:443 10.132.0.14:55138 - v1-completions-path 2026-11-03T20:42:04.380064Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:42:06.803588Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:42:25.745032Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:42:30.401919Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:42:45.679560Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:42:48.211275Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:43:05.721550Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:43:16.438736Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:43:19.363861Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:43:37.617902Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:43:40.657731Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T20:43:47.756Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 37 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "63678a75-dc24-9762-941a-3bca39b0ec70" "maas.apps.e7be3e6863d7d8b1aa59.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.70:443 10.132.0.14:58764 - v1-completions-path 2026-11-03T20:43:47.978236Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T20:43:47.978274Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T20:43:47.978289Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-03T20:43:47.978294Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 [2026-11-03T20:43:47.904Z] "DELETE /v1/api-keys/fffb5423-95b2-4765-a042-0c1cb1b569d2 HTTP/1.1" 200 - via_upstream - "-" 0 375 73 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "ce8713b4-5a01-4a95-a9d3-c2add09ee3ff" "maas.apps.e7be3e6863d7d8b1aa59.prod.konflux-ocp-ci.dev" "10.132.0.50:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.70:39746 10.134.0.70:443 10.132.0.14:58778 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T20:43:49.171Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 76 0 34 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "1e7e2245-2179-4613-bd6b-fb091640770c" "maas.apps.e7be3e6863d7d8b1aa59.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.70:443 10.132.0.14:58788 - v1-completions-path 2026-11-03T20:43:54.708626Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:44:12.394641Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:44:22.944624Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T20:44:39.927Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 41 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "e8787d53-6799-4764-adac-a23f53eb03b7" "maas.apps.e7be3e6863d7d8b1aa59.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.70:443 10.132.0.14:55378 - v1-completions-path 2026-11-03T20:46:22.303463Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:46:25.796286Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:47:57.694526Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:48:13.171101Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:48:15.380647Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:48:34.165347Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:50:00.498931Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T20:50:00.498960Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T20:50:00.498969Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T20:50:00.498974Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-03T20:50:00.429Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 69 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "99fe6550-4ba6-4244-8805-3efefc9defd6" "maas.apps.e7be3e6863d7d8b1aa59.prod.konflux-ocp-ci.dev" "10.132.0.50:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.70:47062 10.134.0.70:443 10.132.0.14:39500 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T20:50:04.112528Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:50:21.930085Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:50:31.017225Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-03T20:50:42.553447Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:50:54.334721Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:50:58.198897Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:51:09.235528Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:51:14.189136Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:51:31.551111Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:54:22.438391Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:54:30.619453Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T20:54:36.654Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 429 - - "-" 61 18 60 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "36f118fa-cc1b-42e6-b749-420ca9f2daab" "maas.apps.e7be3e6863d7d8b1aa59.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.70:443 10.132.0.14:35904 - v1-catch-all-path 2026-11-03T20:54:48.689155Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:54:51.003409Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:55:05.682486Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T20:55:16.677624Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9