2026-10-30T18:50:27.915168Z info FLAG: --concurrency="0" 2026-10-30T18:50:27.915297Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T18:50:27.915320Z info FLAG: --help="false" 2026-10-30T18:50:27.915337Z info FLAG: --log_as_json="false" 2026-10-30T18:50:27.915352Z info FLAG: --log_caller="" 2026-10-30T18:50:27.915367Z info FLAG: --log_output_level="default:info" 2026-10-30T18:50:27.915382Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T18:50:27.915400Z info FLAG: --log_target="[stdout]" 2026-10-30T18:50:27.915416Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T18:50:27.915430Z info FLAG: --outlierLogPath="" 2026-10-30T18:50:27.915445Z info FLAG: --profiling="true" 2026-10-30T18:50:27.915466Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T18:50:27.915482Z info FLAG: --proxyLogLevel="warning" 2026-10-30T18:50:27.915497Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T18:50:27.915512Z info FLAG: --stsPort="0" 2026-10-30T18:50:27.915527Z info FLAG: --templateFile="" 2026-10-30T18:50:27.915542Z info FLAG: --tokenManagerPlugin="" 2026-10-30T18:50:27.915561Z info FLAG: --vklog="0" 2026-10-30T18:50:27.915577Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T18:50:27.915772Z info Proxy role ips=[10.132.0.64] type=router id=maas-default-gateway-openshift-default-68cb9b4b95-p2rlg.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T18:50:27.915849Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T18:50:27.917041Z info cpu limit detected as 2, setting concurrency 2026-10-30T18:50:27.917357Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T18:50:27.917407Z info JWT policy is third-party-jwt 2026-10-30T18:50:27.917426Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T18:50:27.919799Z info Starting default Istio SDS Server 2026-10-30T18:50:27.919828Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T18:50:27.920600Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T18:50:27.920228Z info Opening status port 15020 2026-10-30T18:50:27.921304Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T18:50:27.921521Z info sds Starting SDS grpc server 2026-10-30T18:50:27.921566Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T18:50:27.923222Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T18:50:27.924181Z info Starting proxy agent 2026-10-30T18:50:27.924244Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T18:50:28.016338Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T18:50:28.053812Z info cache generated new workload certificate resourceName=default latency=132.139404ms ttl=23h59m59.946193638s 2026-10-30T18:50:28.053851Z info cache Root cert has changed, start rotating root cert 2026-10-30T18:50:28.053887Z info cache returned workload trust anchor from cache ttl=23h59m59.946113549s 2026-10-30T18:50:28.214080Z info ads ADS: new connection for node:1 2026-10-30T18:50:28.240157Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-10-30T18:50:28.241259Z info cache read certificate from file resource=file-root:system 2026-10-30T18:50:28.287632Z info ads ADS: new connection for node:2 2026-10-30T18:50:28.287708Z info cache returned workload certificate from cache ttl=23h59m59.712294507s 2026-10-30T18:50:28.290680Z info ads ADS: new connection for node:3 2026-10-30T18:50:28.290775Z info cache returned workload trust anchor from cache ttl=23h59m59.709226512s 2026-10-30T18:50:28.426678Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:50:39.322011Z info Readiness succeeded in 11.409702879s 2026-10-30T18:50:39.322441Z info Envoy proxy is ready 2026-10-30T18:50:42.114045Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T18:50:42.118259Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T18:50:42.118314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T18:50:42.118341Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T18:50:42.118649Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T18:50:41.881Z] "POST /anything/e2e-rt-sw-a58f71-client HTTP/1.1" 200 - via_upstream - "-" 124 1760 237 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "e7c05b89-1a4e-4436-9d28-ce11299cc09a" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.64:57040 10.132.0.64:443 10.132.0.17:38098 - llm.external-model-e2e-rt-sw-a58f71-e317ca62b21e.0 2026-10-30T18:50:50.229415Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:51:03.876778Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:51:07.516203Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T18:51:07.516310Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T18:51:07.516354Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T18:51:07.516384Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T18:51:07.516723Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 2026-10-30T18:51:07.714363Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T18:51:07.714470Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T18:51:07.714505Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T18:51:07.714513Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-10-30T18:51:07.714772Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-10-30T18:51:07.380Z] "POST /anything/e2e-rt-eq-92a70c-client HTTP/1.1" 200 - via_upstream - "-" 124 1757 136 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "13c62554-3b88-4b30-93c8-a78de3c7d885" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.64:41384 10.132.0.64:443 10.132.0.17:58556 - llm.external-model-e2e-rt-eq-92a70c-5b7843bce990.0 [2026-10-30T18:51:07.580Z] "POST /anything/e2e-rt-eq-92a70c-client HTTP/1.1" 200 - via_upstream - "-" 124 1764 134 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "c65902ce-0f73-4bab-bc17-ea79999b21f0" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.64:41392 10.132.0.64:443 10.132.0.17:58572 - llm.external-model-e2e-rt-eq-92a70c-5b7843bce990.0 2026-10-30T18:51:12.411172Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T18:51:12.411211Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T18:51:12.411226Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T18:51:12.411232Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T18:51:12.411463Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T18:51:12.293Z] "POST /anything/e2e-rt-eq-92a70c-client HTTP/1.1" 200 - via_upstream - "-" 124 1756 117 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "592148af-bc3a-4954-9b95-dbf08b590e37" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.64:41384 10.132.0.64:443 10.132.0.17:58592 - llm.external-model-e2e-rt-eq-92a70c-5b7843bce990.0 2026-10-30T18:51:38.929900Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:51:38.971208Z info ads ADS: "" 1 terminated 2026-10-30T18:51:39.924468Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:51:56.258409Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:52:38.860899Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:53:01.202839Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:54:44.251110Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:54:51.249471Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T18:54:51.249526Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T18:54:51.249550Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-10-30T18:54:51.249558Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 [2026-10-30T18:54:50.995Z] "DELETE /v1/api-keys/f569dae0-c630-4d88-b473-52a250bca786 HTTP/1.1" 200 - via_upstream - "-" 0 370 253 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "4857fe4e-50f3-48f5-a379-0f07ae18c7c3" "maas.apps.c27b2eb8e6fcb688611a.prod.konflux-ocp-ci.dev" "10.132.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.64:59436 10.132.0.64:443 10.132.0.17:43584 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T18:54:52.715008Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T18:54:52.715042Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T18:54:52.715056Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-10-30T18:54:52.715061Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 [2026-10-30T18:54:52.600Z] "DELETE /v1/api-keys/0ec80dd6-dffc-48b0-9613-7831300485c0 HTTP/1.1" 200 - via_upstream - "-" 0 321 114 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "c9cdbd24-e77b-4287-b73d-b6242f036e2e" "maas.apps.c27b2eb8e6fcb688611a.prod.konflux-ocp-ci.dev" "10.132.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.64:59436 10.132.0.64:443 10.132.0.17:43590 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T18:57:21.850Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 110 326 203 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "fa1847f9-234f-4ed3-822a-ea9e3a833e4e" "maas.apps.c27b2eb8e6fcb688611a.prod.konflux-ocp-ci.dev" "10.133.0.46:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.64:51216 10.132.0.64:443 10.132.0.17:45258 - v1-chat-completions-path 2026-10-30T18:58:48.662666Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T18:58:48.662803Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T18:58:48.662850Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T18:58:48.662898Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T18:58:48.663285Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 2026-10-30T18:58:48.663405Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T18:58:48.663437Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T18:58:48.663459Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T18:58:48.663482Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-10-30T18:58:48.475Z] "POST /v1/chat/completions HTTP/1.1" 404 - via_upstream - "-" 126 129 188 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "09345462-3c40-4d83-b683-298988bcd91d" "maas.apps.c27b2eb8e6fcb688611a.prod.konflux-ocp-ci.dev" "10.133.0.46:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.64:54550 10.132.0.64:443 10.132.0.17:44112 - v1-chat-completions-path 2026-10-30T18:59:53.771290Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-10-30T18:59:53.771747Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T18:59:53.772016Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T18:59:53.685Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 135 1552 86 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "8997838f-e97b-43d1-b3ab-89a03c1ba68b" "maas.apps.c27b2eb8e6fcb688611a.prod.konflux-ocp-ci.dev" "10.133.0.46:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.64:54550 10.132.0.64:443 10.132.0.17:54750 - v1-chat-completions-path 2026-10-30T19:01:03.855647Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: gRPC status code is not OK thread=25 [2026-10-30T19:01:01.839Z] "POST /maas-api/v1/tokens HTTP/1.1" 500 - - "-" 20 23 2017 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "19aa0582-f95d-4ea9-8094-52d7c64929ab" "maas.apps.c27b2eb8e6fcb688611a.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.132.0.64:443 10.132.0.17:33180 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T19:07:53.795408Z info ads ADS: new connection for node:4 2026-10-30T19:07:53.827759Z info cache read certificate from file resource=file-root:system 2026-10-30T19:16:05.977282Z info ads ADS: "" 4 terminated