2026-11-03T21:18:48.868380Z info FLAG: --concurrency="0" 2026-11-03T21:18:48.868397Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T21:18:48.868401Z info FLAG: --help="false" 2026-11-03T21:18:48.868404Z info FLAG: --log_as_json="false" 2026-11-03T21:18:48.868406Z info FLAG: --log_caller="" 2026-11-03T21:18:48.868408Z info FLAG: --log_output_level="default:info" 2026-11-03T21:18:48.868411Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T21:18:48.868431Z info FLAG: --log_target="[stdout]" 2026-11-03T21:18:48.868434Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T21:18:48.868436Z info FLAG: --outlierLogPath="" 2026-11-03T21:18:48.868438Z info FLAG: --profiling="true" 2026-11-03T21:18:48.868440Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T21:18:48.868442Z info FLAG: --proxyLogLevel="warning" 2026-11-03T21:18:48.868444Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T21:18:48.868446Z info FLAG: --stsPort="0" 2026-11-03T21:18:48.868448Z info FLAG: --templateFile="" 2026-11-03T21:18:48.868449Z info FLAG: --tokenManagerPlugin="" 2026-11-03T21:18:48.868452Z info FLAG: --vklog="0" 2026-11-03T21:18:48.868454Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T21:18:48.868583Z info Proxy role ips=[10.134.0.123] type=router id=maas-default-gateway-openshift-default-696f47c6c-rh44r.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T21:18:48.868628Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T21:18:48.869644Z info cpu limit detected as 2, setting concurrency 2026-11-03T21:18:48.869837Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T21:18:48.869848Z info JWT policy is third-party-jwt 2026-11-03T21:18:48.869850Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T21:18:48.871441Z info Starting default Istio SDS Server 2026-11-03T21:18:48.871459Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T21:18:48.871475Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T21:18:48.871864Z info Opening status port 15020 2026-11-03T21:18:48.872962Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T21:18:48.873338Z info sds Starting SDS grpc server 2026-11-03T21:18:48.873352Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T21:18:48.874250Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T21:18:48.874993Z info Starting proxy agent 2026-11-03T21:18:48.875021Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T21:18:48.942860Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T21:18:49.027975Z info cache generated new workload certificate resourceName=default latency=154.506353ms ttl=23h59m59.972030072s 2026-11-03T21:18:49.028013Z info cache Root cert has changed, start rotating root cert 2026-11-03T21:18:49.028049Z info cache returned workload trust anchor from cache ttl=23h59m59.97195183s 2026-11-03T21:18:49.110328Z info ads ADS: new connection for node:1 2026-11-03T21:18:49.110406Z info cache returned workload trust anchor from cache ttl=23h59m59.889595263s 2026-11-03T21:18:49.110456Z info ads ADS: new connection for node:2 2026-11-03T21:18:49.110507Z info cache returned workload certificate from cache ttl=23h59m59.889494463s 2026-11-03T21:18:49.202799Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:18:57.517398Z info Readiness succeeded in 8.651530067s 2026-11-03T21:18:57.517829Z info Envoy proxy is ready 2026-11-03T21:19:27.780062Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T21:19:27.780086Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T21:19:27.780090Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T21:19:27.780093Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-11-03T21:19:27.721Z] "POST /llm/premium-simulated-simulated-premium/v1/completions HTTP/1.1" 403 - - "-" 74 77 58 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "41a90937-db66-4866-900a-32b156dd5d6f" "maas.apps.2cc86969c691a66e5e58.prod.konflux-ocp-ci.dev" "-" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.123:443 10.132.0.18:58016 - v1-completions-path 2026-11-03T21:19:30.565796Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T21:19:30.565825Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T21:19:30.565828Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T21:19:30.565832Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-11-03T21:19:30.524Z] "POST /llm/premium-simulated-simulated-premium/v1/completions HTTP/1.1" 403 - - "-" 74 77 41 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "9da4c1c7-aa31-44c3-8b9a-54813f70fd6b" "maas.apps.2cc86969c691a66e5e58.prod.konflux-ocp-ci.dev" "-" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.123:443 10.132.0.18:38552 - v1-completions-path 2026-11-03T21:19:37.908298Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:19:43.098820Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:19:43.098854Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:19:43.098866Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:19:43.098872Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T21:19:42.979Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 87 329 119 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "e95c195b-38be-4843-ab47-529829291d12" "maas.apps.2cc86969c691a66e5e58.prod.konflux-ocp-ci.dev" "10.133.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.123:54634 10.134.0.123:443 10.132.0.18:35928 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:19:58.802452Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:20:13.174078Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:20:35.190088Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:20:55.491242Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:21:11.942899Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:21:19.567Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 79 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "5906b1c1-0768-46c2-9830-cf5eb027acb3" "maas.apps.2cc86969c691a66e5e58.prod.konflux-ocp-ci.dev" "10.132.0.61:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.123:51418 10.134.0.123:443 10.132.0.18:39196 - v1-completions-path [2026-11-03T21:21:19.952Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 61 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "284136fb-a998-4217-b373-116b66efabd8" "maas.apps.2cc86969c691a66e5e58.prod.konflux-ocp-ci.dev" "10.132.0.61:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.123:51422 10.134.0.123:443 10.132.0.18:39206 - v1-completions-path [2026-11-03T21:21:23.041Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 60 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "f744210c-ca17-43fe-a137-319fc881a376" "maas.apps.2cc86969c691a66e5e58.prod.konflux-ocp-ci.dev" "10.132.0.61:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.123:51422 10.134.0.123:443 10.132.0.18:39222 - v1-completions-path 2026-11-03T21:21:43.086127Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:21:53.745841Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:21:59.829209Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:22:14.161Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 60 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "6782fbbd-cbd4-4174-b5f1-9749d0e39d32" "maas.apps.2cc86969c691a66e5e58.prod.konflux-ocp-ci.dev" "10.132.0.61:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.123:51418 10.134.0.123:443 10.132.0.18:58228 - v1-completions-path 2026-11-03T21:22:15.222975Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-03T21:22:15.167Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 59 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "f37624f1-9a3a-464f-ac62-cc87c027bf70" "maas.apps.2cc86969c691a66e5e58.prod.konflux-ocp-ci.dev" "10.132.0.61:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.123:51418 10.134.0.123:443 10.132.0.18:58232 - v1-completions-path 2026-11-03T21:22:28.652688Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:22:33.071722Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:22:42.945Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 74 300 78 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "8194f6be-8e01-44c2-9b21-148939d85c3e" "maas.apps.2cc86969c691a66e5e58.prod.konflux-ocp-ci.dev" "10.132.0.63:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.134.0.123:41780 10.134.0.123:443 10.132.0.18:58166 - v1-completions-path 2026-11-03T21:22:56.352268Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:23:02.342522Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:23:28.621123Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:23:47.566110Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:24:00.416779Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:24:34.559929Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:25:09.433955Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:25:19.973014Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:29:04.930935Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:29:04.930967Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:29:04.930981Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:29:04.930986Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T21:29:04.804Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 126 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "d1a7b18b-345f-4ff9-96ea-2e95551f3b27" "maas.apps.2cc86969c691a66e5e58.prod.konflux-ocp-ci.dev" "10.133.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.123:44994 10.134.0.123:443 10.132.0.18:57870 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:29:28.922276Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:29:46.010958Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:29:58.607043Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:30:18.402142Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:32:21.765298Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:33:02.606999Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:33:29.573521Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:33:45.444348Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:34:04.331051Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:34:16.920583Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:34:21.521531Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:34:33.404Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 59 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "ef0a268d-68dc-417e-afa7-0e7edf09dd9d" "maas.apps.2cc86969c691a66e5e58.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.123:443 10.132.0.18:42400 - v1-completions-path 2026-11-03T21:34:47.508491Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:38:29.531885Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:38:36.697477Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-11-03T21:38:36.620Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 77 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "ea925be2-1d22-4497-9eb5-2cfd7e87fb6f" "maas.apps.2cc86969c691a66e5e58.prod.konflux-ocp-ci.dev" "10.132.0.60:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.123:35874 10.134.0.123:443 10.132.0.18:54984 - v1-catch-all-path 2026-11-03T21:38:40.991386Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:39:01.865990Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:39:06.424521Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:39:22.670848Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:39:34.507462Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8