2026-11-03T23:12:50.344522Z info FLAG: --concurrency="0" 2026-11-03T23:12:50.344540Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T23:12:50.344544Z info FLAG: --help="false" 2026-11-03T23:12:50.344547Z info FLAG: --log_as_json="false" 2026-11-03T23:12:50.344549Z info FLAG: --log_caller="" 2026-11-03T23:12:50.344551Z info FLAG: --log_output_level="default:info" 2026-11-03T23:12:50.344553Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T23:12:50.344557Z info FLAG: --log_target="[stdout]" 2026-11-03T23:12:50.344559Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T23:12:50.344561Z info FLAG: --outlierLogPath="" 2026-11-03T23:12:50.344563Z info FLAG: --profiling="true" 2026-11-03T23:12:50.344566Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T23:12:50.344568Z info FLAG: --proxyLogLevel="warning" 2026-11-03T23:12:50.344570Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T23:12:50.344572Z info FLAG: --stsPort="0" 2026-11-03T23:12:50.344574Z info FLAG: --templateFile="" 2026-11-03T23:12:50.344576Z info FLAG: --tokenManagerPlugin="" 2026-11-03T23:12:50.344579Z info FLAG: --vklog="0" 2026-11-03T23:12:50.344581Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T23:12:50.344731Z info Proxy role ips=[10.133.1.18] type=router id=maas-default-gateway-openshift-default-59f957fdd7-qrmft.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T23:12:50.344787Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T23:12:50.345935Z info cpu limit detected as 2, setting concurrency 2026-11-03T23:12:50.346184Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T23:12:50.346195Z info JWT policy is third-party-jwt 2026-11-03T23:12:50.346198Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T23:12:50.348167Z info Starting default Istio SDS Server 2026-11-03T23:12:50.348197Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T23:12:50.348229Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T23:12:50.348521Z info Opening status port 15020 2026-11-03T23:12:50.349309Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T23:12:50.350554Z info sds Starting SDS grpc server 2026-11-03T23:12:50.350582Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T23:12:50.350866Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T23:12:50.351772Z info Starting proxy agent 2026-11-03T23:12:50.351825Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T23:12:50.414740Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T23:12:50.528159Z info cache generated new workload certificate resourceName=default latency=177.49922ms ttl=23h59m59.471845475s 2026-11-03T23:12:50.528201Z info cache Root cert has changed, start rotating root cert 2026-11-03T23:12:50.528248Z info cache returned workload trust anchor from cache ttl=23h59m59.471753056s 2026-11-03T23:12:50.562920Z info ads ADS: new connection for node:1 2026-11-03T23:12:50.562987Z info cache returned workload certificate from cache ttl=23h59m59.43701421s 2026-11-03T23:12:50.563840Z info ads ADS: new connection for node:2 2026-11-03T23:12:50.564048Z info cache returned workload trust anchor from cache ttl=23h59m59.435953727s 2026-11-03T23:12:50.643284Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:12:58.517695Z info Readiness succeeded in 8.176074777s 2026-11-03T23:12:58.518270Z info Envoy proxy is ready 2026-11-03T23:13:33.256213Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:13:54.733264Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:13:59.605641Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:14:26.280971Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:14:43.167533Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T23:14:51.527Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 84 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "4caee832-5cb7-4766-9ea9-2e2c11a58310" "maas.apps.d5a87148a14ca6a443cd.prod.konflux-ocp-ci.dev" "10.133.0.28:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.18:33256 10.133.1.18:443 10.132.0.13:48150 - v1-completions-path 2026-11-03T23:15:14.135228Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:15:24.897574Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:15:31.315727Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T23:15:48.635Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 63 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "efbf6caf-6570-4850-b73f-5965b4212ac1" "maas.apps.d5a87148a14ca6a443cd.prod.konflux-ocp-ci.dev" "10.133.0.28:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.18:33256 10.133.1.18:443 10.132.0.13:47318 - v1-completions-path 2026-11-03T23:16:02.697500Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:16:07.378327Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:16:31.243900Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:16:37.990383Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:17:03.689524Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:17:26.244603Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:17:39.349708Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:17:45.531679Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T23:18:08.865Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 39 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "ce00cae4-d34e-4dec-9c22-eb267fabe269" "maas.apps.d5a87148a14ca6a443cd.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.1.18:443 10.132.0.13:44488 - v1-completions-path 2026-11-03T23:18:13.164039Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:18:18.853932Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:18:27.786781Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:18:27.786851Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:18:27.786869Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:18:27.786876Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T23:18:27.671Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 89 332 115 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "249fbb3a-68a8-46bd-a774-648a8f95d908" "maas.apps.d5a87148a14ca6a443cd.prod.konflux-ocp-ci.dev" "10.134.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.1.18:35166 10.133.1.18:443 10.132.0.13:45368 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T23:18:28.934271Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:18:28.934311Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:18:28.934327Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:18:28.934334Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T23:18:28.826Z] "DELETE /v1/api-keys/edb5fd4a-9052-46c3-bd19-87453a41fd59 HTTP/1.1" 200 - via_upstream - "-" 0 375 108 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "0c6ba8d9-bc7f-4a82-b504-d9f926e0d624" "maas.apps.d5a87148a14ca6a443cd.prod.konflux-ocp-ci.dev" "10.134.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.1.18:35166 10.133.1.18:443 10.132.0.13:45386 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T23:18:38.694821Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:18:47.793968Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-11-03T23:18:47.732Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 299 63 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "1eb7165f-768a-4d21-a0b0-2467bf230142" "maas.apps.d5a87148a14ca6a443cd.prod.konflux-ocp-ci.dev" "10.133.0.28:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.18:33256 10.133.1.18:443 10.132.0.13:41692 - v1-completions-path 2026-11-03T23:19:01.693130Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:19:12.277564Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:19:51.055661Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:19:57.074293Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:24:05.797979Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:24:23.222509Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:24:27.964283Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:24:41.687016Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T23:24:41.687043Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T23:24:41.687047Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T23:24:41.687052Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-11-03T23:24:41.651Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 123 35 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "75205cd7-a02f-432f-9197-3190f7f8bfe2" "maas.apps.d5a87148a14ca6a443cd.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.1.18:443 10.132.0.13:38770 - v1-completions-path 2026-11-03T23:24:55.741091Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:27:02.329316Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:27:47.615075Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:28:18.411224Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:28:38.262924Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:28:43.533378Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:29:02.972328Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:29:14.334560Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:29:19.352332Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:29:49.190909Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:33:30.391856Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:33:42.298232Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:33:42.298265Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:33:42.298276Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:33:42.298281Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T23:33:42.986390Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-11-03T23:33:42.179Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 324 118 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "1791338f-7935-4449-a488-ab80af36705a" "maas.apps.d5a87148a14ca6a443cd.prod.konflux-ocp-ci.dev" "10.132.0.155:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.1.18:44130 10.133.1.18:443 10.132.0.13:49132 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T23:33:42.910Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 76 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "b6c31cfc-9d2a-4e19-8d2a-0a06fb1db6cf" "maas.apps.d5a87148a14ca6a443cd.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.1.18:40680 10.133.1.18:443 10.132.0.13:49146 - v1-catch-all-path 2026-11-03T23:33:48.892070Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T23:33:57.939Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 429 - - "-" 61 18 57 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "90c353e7-855d-4981-8448-da6015604682" "maas.apps.d5a87148a14ca6a443cd.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.1.18:443 10.132.0.13:55870 - v1-catch-all-path 2026-11-03T23:34:12.164563Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:34:16.860495Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:34:31.301786Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:34:42.793902Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8