2026-11-02T20:21:03.102428Z info FLAG: --concurrency="0" 2026-11-02T20:21:03.102446Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-02T20:21:03.102450Z info FLAG: --help="false" 2026-11-02T20:21:03.102453Z info FLAG: --log_as_json="false" 2026-11-02T20:21:03.102455Z info FLAG: --log_caller="" 2026-11-02T20:21:03.102457Z info FLAG: --log_output_level="default:info" 2026-11-02T20:21:03.102459Z info FLAG: --log_stacktrace_level="default:none" 2026-11-02T20:21:03.102463Z info FLAG: --log_target="[stdout]" 2026-11-02T20:21:03.102466Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-02T20:21:03.102468Z info FLAG: --outlierLogPath="" 2026-11-02T20:21:03.102469Z info FLAG: --profiling="true" 2026-11-02T20:21:03.102472Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-02T20:21:03.102473Z info FLAG: --proxyLogLevel="warning" 2026-11-02T20:21:03.102476Z info FLAG: --serviceCluster="istio-proxy" 2026-11-02T20:21:03.102478Z info FLAG: --stsPort="0" 2026-11-02T20:21:03.102480Z info FLAG: --templateFile="" 2026-11-02T20:21:03.102482Z info FLAG: --tokenManagerPlugin="" 2026-11-02T20:21:03.102484Z info FLAG: --vklog="0" 2026-11-02T20:21:03.102487Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-02T20:21:03.102622Z info Proxy role ips=[10.133.0.134] type=router id=maas-default-gateway-openshift-default-75fd9f5d47-dw7ld.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-02T20:21:03.102669Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-02T20:21:03.103662Z info cpu limit detected as 2, setting concurrency 2026-11-02T20:21:03.103903Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-02T20:21:03.103915Z info JWT policy is third-party-jwt 2026-11-02T20:21:03.103917Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-02T20:21:03.105851Z info Starting default Istio SDS Server 2026-11-02T20:21:03.105882Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-02T20:21:03.105905Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-02T20:21:03.106149Z info Opening status port 15020 2026-11-02T20:21:03.107256Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-02T20:21:03.107427Z info sds Starting SDS grpc server 2026-11-02T20:21:03.107441Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-02T20:21:03.108534Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-02T20:21:03.109279Z info Starting proxy agent 2026-11-02T20:21:03.109314Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-02T20:21:03.170631Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-02T20:21:03.253915Z info cache generated new workload certificate resourceName=default latency=146.384465ms ttl=23h59m59.746088465s 2026-11-02T20:21:03.253952Z info cache Root cert has changed, start rotating root cert 2026-11-02T20:21:03.253982Z info cache returned workload trust anchor from cache ttl=23h59m59.746018981s 2026-11-02T20:21:03.321937Z info ads ADS: new connection for node:1 2026-11-02T20:21:03.322036Z info cache returned workload certificate from cache ttl=23h59m59.677966498s 2026-11-02T20:21:03.322792Z info ads ADS: new connection for node:2 2026-11-02T20:21:03.322858Z info cache returned workload trust anchor from cache ttl=23h59m59.67714321s 2026-11-02T20:21:03.402009Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:21:11.314484Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:21:12.256811Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-02T20:21:12.580170Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-02T20:21:13.258644Z info Readiness succeeded in 10.15883463s 2026-11-02T20:21:13.259044Z info Envoy proxy is ready [2026-11-02T20:21:32.874Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 295 101 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "5b189b21-7361-43fc-8381-2be3c85b27d5" "maas.apps.489062d6ac89ec9ac5eb.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.134:53848 10.133.0.134:443 10.132.0.21:51398 - v1-completions-path 2026-11-02T20:21:37.736110Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:21:52.592362Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-02T20:22:02.500Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 71 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "b1811f1e-883b-47a3-9194-9ad253820a1c" "maas.apps.489062d6ac89ec9ac5eb.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.134:34322 10.133.0.134:443 10.132.0.21:46064 - v1-completions-path [2026-11-02T20:22:03.497Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 68 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "62201913-5d87-4e3d-b839-b5315550e2e8" "maas.apps.489062d6ac89ec9ac5eb.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.134:34338 10.133.0.134:443 10.132.0.21:46072 - v1-completions-path 2026-11-02T20:22:05.607276Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-02T20:22:05.521Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 86 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "3505d74c-f2e5-4dca-9c1c-8f9d18a0d42c" "maas.apps.489062d6ac89ec9ac5eb.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.134:34322 10.133.0.134:443 10.132.0.21:46088 - v1-completions-path 2026-11-02T20:22:25.414997Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:22:36.050465Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:22:42.188525Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-02T20:22:54.279Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 60 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "e4b283c4-c39f-4373-81f8-8bf2870ce8ed" "maas.apps.489062d6ac89ec9ac5eb.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.134:34322 10.133.0.134:443 10.132.0.21:38826 - v1-completions-path [2026-11-02T20:22:56.605Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 58 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "ce933b64-bc3c-44b6-afe5-261128a5cd89" "maas.apps.489062d6ac89ec9ac5eb.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.134:34338 10.133.0.134:443 10.132.0.21:38836 - v1-completions-path 2026-11-02T20:23:10.607328Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:23:14.944930Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:23:38.670620Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:23:44.935083Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:24:10.772052Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:24:29.610180Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:24:42.623985Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:25:07.045484Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T20:25:07.045523Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T20:25:07.045540Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T20:25:07.045546Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-02T20:25:06.934Z] "DELETE /v1/api-keys/ef840740-4ec9-4c62-9309-9c58f49db7c9 HTTP/1.1" 404 - via_upstream - "-" 0 29 110 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "a81679ad-623e-4482-8f11-e4ee7292e78c" "maas.apps.489062d6ac89ec9ac5eb.prod.konflux-ocp-ci.dev" "10.133.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.134:48066 10.133.0.134:443 10.132.0.21:47110 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T20:25:15.261313Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:25:37.050726Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:25:47.516786Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:29:52.898518Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:30:09.977125Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:30:14.278903Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:30:38.852739Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:33:21.916360Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:33:48.791517Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:34:14.346604Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:34:29.610618Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:34:35.943568Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:34:47.350417Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:34:50.275687Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:35:13.318168Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:38:40.208617Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-02T20:38:40.208641Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-02T20:38:40.208645Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-02T20:38:40.208649Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-11-02T20:38:40.059Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 403 - - "-" 61 103 149 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "a9cb59fc-305c-47e2-a5b8-7c68cc175fab" "maas.apps.489062d6ac89ec9ac5eb.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.134:443 10.132.0.21:49194 - v1-catch-all-path 2026-11-02T20:38:48.248688Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:38:58.605478Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:39:19.559645Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:39:23.757312Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:39:31.882442Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T20:39:31.882482Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T20:39:31.882498Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T20:39:31.882505Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-02T20:39:31.771Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 73 316 110 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "9fd04e50-5c34-49a2-88d6-e880f6b67625" "maas.apps.489062d6ac89ec9ac5eb.prod.konflux-ocp-ci.dev" "10.134.0.177:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.134:48580 10.133.0.134:443 10.132.0.21:42654 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T20:39:45.733902Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T20:39:57.325832Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9