2026-11-02T06:40:38.129052Z info FLAG: --concurrency="0" 2026-11-02T06:40:38.129074Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-02T06:40:38.129079Z info FLAG: --help="false" 2026-11-02T06:40:38.129083Z info FLAG: --log_as_json="false" 2026-11-02T06:40:38.129086Z info FLAG: --log_caller="" 2026-11-02T06:40:38.129089Z info FLAG: --log_output_level="default:info" 2026-11-02T06:40:38.129092Z info FLAG: --log_stacktrace_level="default:none" 2026-11-02T06:40:38.129099Z info FLAG: --log_target="[stdout]" 2026-11-02T06:40:38.129102Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-02T06:40:38.129105Z info FLAG: --outlierLogPath="" 2026-11-02T06:40:38.129109Z info FLAG: --profiling="true" 2026-11-02T06:40:38.129112Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-02T06:40:38.129118Z info FLAG: --proxyLogLevel="warning" 2026-11-02T06:40:38.129121Z info FLAG: --serviceCluster="istio-proxy" 2026-11-02T06:40:38.129124Z info FLAG: --stsPort="0" 2026-11-02T06:40:38.129127Z info FLAG: --templateFile="" 2026-11-02T06:40:38.129129Z info FLAG: --tokenManagerPlugin="" 2026-11-02T06:40:38.129133Z info FLAG: --vklog="0" 2026-11-02T06:40:38.129137Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-02T06:40:38.129327Z info Proxy role ips=[10.134.0.102] type=router id=maas-default-gateway-openshift-default-55f85cf5d7-6mt9r.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-02T06:40:38.129384Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-02T06:40:38.130513Z info cpu limit detected as 2, setting concurrency 2026-11-02T06:40:38.130732Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-02T06:40:38.130745Z info JWT policy is third-party-jwt 2026-11-02T06:40:38.130749Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-02T06:40:38.132734Z info Starting default Istio SDS Server 2026-11-02T06:40:38.132757Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-02T06:40:38.132774Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-02T06:40:38.133141Z info Opening status port 15020 2026-11-02T06:40:38.134048Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-02T06:40:38.134796Z info sds Starting SDS grpc server 2026-11-02T06:40:38.134888Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-02T06:40:38.135426Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-02T06:40:38.136222Z info Starting proxy agent 2026-11-02T06:40:38.136259Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-02T06:40:38.205941Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-02T06:40:38.275428Z info cache generated new workload certificate resourceName=default latency=140.442306ms ttl=23h59m59.7245772s 2026-11-02T06:40:38.275464Z info cache Root cert has changed, start rotating root cert 2026-11-02T06:40:38.275497Z info cache returned workload trust anchor from cache ttl=23h59m59.724503435s 2026-11-02T06:40:38.413650Z info ads ADS: new connection for node:1 2026-11-02T06:40:38.413726Z info cache returned workload certificate from cache ttl=23h59m59.586275495s 2026-11-02T06:40:38.414335Z info ads ADS: new connection for node:2 2026-11-02T06:40:38.414464Z info cache returned workload trust anchor from cache ttl=23h59m59.585536944s 2026-11-02T06:40:38.506334Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T06:40:46.880212Z info Readiness succeeded in 8.753513667s 2026-11-02T06:40:46.880947Z info Envoy proxy is ready 2026-11-02T06:40:51.307138Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T06:42:11.527798Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T06:47:41.717859Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T06:49:11.052353Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T06:53:37.926925Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T06:55:13.684796Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:03:16.661332Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:03:45.031084Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:04:39.598113Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T07:04:47.360Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 67 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "fd66f69c-60da-4954-b0ce-8f64df054852" "maas.apps.d46096a673f3dec2b42d.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.102:443 10.132.0.17:42192 - v1-completions-path 2026-11-02T07:05:00.986190Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:05:05.491204Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:05:17.362810Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T07:05:17.362847Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T07:05:17.362861Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T07:05:17.362866Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T07:05:17.242Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 336 120 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "9e270a70-aeca-423d-9606-86d842e67d25" "maas.apps.d46096a673f3dec2b42d.prod.konflux-ocp-ci.dev" "10.133.0.23:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.102:40766 10.134.0.102:443 10.132.0.17:57930 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T07:05:35.226773Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:05:48.529844Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T07:05:56.767Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 69 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "42142329-cf7d-4565-b673-224659ee47fc" "maas.apps.d46096a673f3dec2b42d.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.102:33608 10.134.0.102:443 10.132.0.17:50230 - v1-completions-path 2026-11-02T07:06:00.140496Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-11-02T07:06:00.084Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 57 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "e7996c7a-a03a-47eb-9d9e-1e5331daf16e" "maas.apps.d46096a673f3dec2b42d.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.102:33608 10.134.0.102:443 10.132.0.17:46224 - v1-completions-path [2026-11-02T07:06:00.914Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 65 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "5d941bac-ae6d-4bf8-8f65-11a77b661cf5" "maas.apps.d46096a673f3dec2b42d.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.102:39686 10.134.0.102:443 10.132.0.17:46236 - v1-completions-path 2026-11-02T07:06:19.503837Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:06:30.076230Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:06:36.239722Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T07:06:49.248Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 64 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "7eab4248-11d5-44ea-82ed-5d0d62197ca8" "maas.apps.d46096a673f3dec2b42d.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.102:39686 10.134.0.102:443 10.132.0.17:35364 - v1-completions-path 2026-11-02T07:07:05.022444Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:07:09.570065Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:07:32.879933Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:07:38.994710Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:08:01.132923Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-02T07:08:01.053Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 80 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "92aa1069-ce8f-439a-98b4-d20cc341b474" "maas.apps.d46096a673f3dec2b42d.prod.konflux-ocp-ci.dev" "10.133.0.30:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.102:45976 10.134.0.102:443 10.132.0.17:56544 - v1-completions-path 2026-11-02T07:08:05.198601Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T07:08:22.013Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 281 67 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "0890a651-d1d6-45e4-9049-24eee9ad811c" "maas.apps.d46096a673f3dec2b42d.prod.konflux-ocp-ci.dev" "10.133.0.30:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.102:37186 10.134.0.102:443 10.132.0.17:49066 - v1-completions-path 2026-11-02T07:08:24.256292Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:08:37.213285Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:08:42.859475Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:09:07.822760Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:09:13.216091Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:09:23.778896Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T07:09:23.778937Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T07:09:23.778954Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T07:09:23.778959Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-02T07:09:23.675Z] "DELETE /v1/api-keys/cdeb5ff7-fc62-49af-bae9-2cc67c913c11 HTTP/1.1" 200 - via_upstream - "-" 0 375 104 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "a8acbeef-982b-4461-adfc-af09d17ed101" "maas.apps.d46096a673f3dec2b42d.prod.konflux-ocp-ci.dev" "10.133.0.23:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.102:32848 10.134.0.102:443 10.132.0.17:56026 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T07:09:31.732455Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:09:53.434099Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:10:03.947326Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:10:22.708065Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:10:28.263086Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:10:30.288559Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-02T07:12:53.656155Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:12:59.426575Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:14:38.229824Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:14:55.419772Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:15:00.153353Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:15:24.362585Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:17:28.519466Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:18:08.423221Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:18:36.677669Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:18:51.892036Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:19:04.905388Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:19:17.001818Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:19:21.846508Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:19:48.788074Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:23:17.992108Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:23:35.279449Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:23:56.173393Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:24:00.972027Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:24:01.715219Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-11-02T07:24:01.620Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 95 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "f539b8e0-0635-473a-a908-980d9a3cd044" "maas.apps.d46096a673f3dec2b42d.prod.konflux-ocp-ci.dev" "10.133.0.27:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.102:34168 10.134.0.102:443 10.132.0.17:45098 - v1-catch-all-path 2026-11-02T07:24:15.546161Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T07:24:26.868254Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8