2026-10-30T18:42:06.823312Z info FLAG: --concurrency="0" 2026-10-30T18:42:06.823336Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T18:42:06.823340Z info FLAG: --help="false" 2026-10-30T18:42:06.823343Z info FLAG: --log_as_json="false" 2026-10-30T18:42:06.823345Z info FLAG: --log_caller="" 2026-10-30T18:42:06.823346Z info FLAG: --log_output_level="default:info" 2026-10-30T18:42:06.823348Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T18:42:06.823353Z info FLAG: --log_target="[stdout]" 2026-10-30T18:42:06.823355Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T18:42:06.823357Z info FLAG: --outlierLogPath="" 2026-10-30T18:42:06.823359Z info FLAG: --profiling="true" 2026-10-30T18:42:06.823361Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T18:42:06.823362Z info FLAG: --proxyLogLevel="warning" 2026-10-30T18:42:06.823365Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T18:42:06.823367Z info FLAG: --stsPort="0" 2026-10-30T18:42:06.823369Z info FLAG: --templateFile="" 2026-10-30T18:42:06.823371Z info FLAG: --tokenManagerPlugin="" 2026-10-30T18:42:06.823373Z info FLAG: --vklog="0" 2026-10-30T18:42:06.823375Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T18:42:06.823510Z info Proxy role ips=[10.134.0.37] type=router id=maas-default-gateway-openshift-default-855f9574cf-pb8bs.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T18:42:06.823553Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T18:42:06.824627Z info cpu limit detected as 2, setting concurrency 2026-10-30T18:42:06.824828Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T18:42:06.824839Z info JWT policy is third-party-jwt 2026-10-30T18:42:06.824842Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T18:42:06.826333Z info Starting default Istio SDS Server 2026-10-30T18:42:06.826359Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T18:42:06.826375Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T18:42:06.826781Z info Opening status port 15020 2026-10-30T18:42:06.827720Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T18:42:06.828390Z info sds Starting SDS grpc server 2026-10-30T18:42:06.828425Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T18:42:06.829254Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T18:42:06.830205Z info Starting proxy agent 2026-10-30T18:42:06.830241Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T18:42:06.893438Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T18:42:06.903524Z info cache generated new workload certificate resourceName=default latency=74.982744ms ttl=23h59m59.096478968s 2026-10-30T18:42:06.903563Z info cache Root cert has changed, start rotating root cert 2026-10-30T18:42:06.903598Z info cache returned workload trust anchor from cache ttl=23h59m59.096403596s 2026-10-30T18:42:06.996229Z info ads ADS: new connection for node:1 2026-10-30T18:42:06.996285Z info cache returned workload certificate from cache ttl=23h59m59.003718044s 2026-10-30T18:42:06.996480Z info ads ADS: new connection for node:2 2026-10-30T18:42:06.996557Z info cache returned workload trust anchor from cache ttl=23h59m59.003444212s 2026-10-30T18:42:07.076750Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:42:14.956807Z info Readiness succeeded in 8.136182192s 2026-10-30T18:42:14.957257Z info Envoy proxy is ready [2026-10-30T18:42:15.341Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 318 90 - "52.71.36.72,10.132.0.19" "curl/7.76.1" "ffed60f3-dded-41d2-8063-a52e84b6aa2e" "maas.apps.3ad58fbeb6df127dd4c4.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.37:44414 10.134.0.37:443 10.132.0.19:47720 - v1-model-routing [2026-10-30T18:42:15.637Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 358 61 - "52.71.36.72,10.132.0.19" "curl/7.76.1" "72931bf5-7f8a-4679-ada1-01ad1d5f7b1d" "maas.apps.3ad58fbeb6df127dd4c4.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.37:44418 10.134.0.37:443 10.132.0.19:47730 - v1-model-routing [2026-10-30T18:42:15.893Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 54 - "52.71.36.72,10.132.0.19" "curl/7.76.1" "6275a857-9c7c-4abf-a7dc-3f2f449a7029" "maas.apps.3ad58fbeb6df127dd4c4.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.37:443 10.132.0.19:48068 - v1-model-routing 2026-10-30T18:42:16.636783Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T18:42:16.636810Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T18:42:16.636813Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T18:42:16.636817Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-10-30T18:42:16.257Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 54 - "52.71.36.72,10.132.0.19" "curl/7.76.1" "2492e2a3-d56e-4617-82a7-815c72970aee" "maas.apps.3ad58fbeb6df127dd4c4.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.37:443 10.132.0.19:48072 - v1-model-routing [2026-10-30T18:42:16.385Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 50 - "52.71.36.72,10.132.0.19" "curl/7.76.1" "4b01f81c-3c48-43b7-a234-1004db6479b9" "maas.apps.3ad58fbeb6df127dd4c4.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.37:443 10.132.0.19:48084 - v1-model-routing [2026-10-30T18:42:16.619Z] "POST /v1/chat/completions HTTP/1.1" 401 - - "-" 122 88 17 - "52.71.36.72,10.132.0.19" "curl/7.76.1" "7932c93f-e1fe-49a7-a93d-7bd92eb7e9cb" "maas.apps.3ad58fbeb6df127dd4c4.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.37:443 10.132.0.19:48096 - v1-model-routing 2026-10-30T18:42:17.201366Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T18:42:17.201404Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T18:42:17.201420Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T18:42:17.201426Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T18:42:17.187Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 91 14 - "52.71.36.72,10.132.0.19" "curl/7.76.1" "1ce790f6-bfab-4933-a8da-dfca4b4e351e" "maas.apps.3ad58fbeb6df127dd4c4.prod.konflux-ocp-ci.dev" "10.133.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.37:36658 10.134.0.37:443 10.132.0.19:48106 - odh-ai-gateway-infra.maas-api-route.4 2026-10-30T18:45:27.576265Z info ads ADS: new connection for node:3 2026-10-30T18:45:27.581576Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-10-30T18:45:27.581621Z info cache read certificate from file resource=file-root:system 2026-10-30T18:45:28.143845Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:45:35.342398Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:45:44.781609Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:46:02.945284Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:46:28.262165Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T18:46:28.262196Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T18:46:28.262206Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T18:46:28.262210Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T18:46:28.262380Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-10-30T18:46:28.126Z] "POST /anything/e2e-rt-eq-1e8613-client HTTP/1.1" 200 - via_upstream - "-" 124 1772 136 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "e6b7a490-bc0c-4f61-a028-3ca59bdc26ff" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.37:53794 10.134.0.37:443 10.132.0.19:57132 - llm.external-model-e2e-rt-eq-1e8613-4d38146b8aab.0 2026-10-30T18:46:57.339413Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:47:00.221789Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:47:00.352246Z info ads ADS: "" 3 terminated 2026-10-30T18:47:14.217977Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:50:38.887469Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T18:50:38.887506Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T18:50:38.887520Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T18:50:38.887525Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T18:50:38.786Z] "DELETE /v1/api-keys/f7f21aa0-23b3-44bd-a436-c72838e87ff1 HTTP/1.1" 200 - via_upstream - "-" 0 321 100 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "8637aa08-7127-4ca3-a06f-f16b68d35c4f" "maas.apps.3ad58fbeb6df127dd4c4.prod.konflux-ocp-ci.dev" "10.133.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.37:48622 10.134.0.37:443 10.132.0.19:47344 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T18:50:40.382522Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T18:50:40.382564Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T18:50:40.382583Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T18:50:40.382589Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T18:50:40.319Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 62 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "ece8e90b-d081-4411-a434-171efc971e6f" "maas.apps.3ad58fbeb6df127dd4c4.prod.konflux-ocp-ci.dev" "10.133.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.37:48632 10.134.0.37:443 10.132.0.19:47356 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T18:52:59.771867Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T18:55:02.552940Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:05:19.605678Z info ads ADS: new connection for node:4 2026-10-30T19:05:19.611655Z info cache read certificate from file resource=file-root:system 2026-10-30T19:13:31.354803Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:13:47.991295Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:14:26.062494Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:15:16.207233Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-10-30T19:18:59.255350Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:19:01.344771Z info ads ADS: new connection for node:5 2026-10-30T19:19:01.345059Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T19:19:01.345135Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T19:19:04.471370Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:19:07.659069Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:19:09.464488Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:19:10.856316Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:19:34.294866Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:21:39.838325Z info ads ADS: "" 4 terminated 2026-10-30T19:22:29.221100Z info ads ADS: "" 5 terminated with stream closed