2026-11-03T19:38:45.021252Z info FLAG: --concurrency="0" 2026-11-03T19:38:45.021271Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T19:38:45.021275Z info FLAG: --help="false" 2026-11-03T19:38:45.021277Z info FLAG: --log_as_json="false" 2026-11-03T19:38:45.021279Z info FLAG: --log_caller="" 2026-11-03T19:38:45.021281Z info FLAG: --log_output_level="default:info" 2026-11-03T19:38:45.021283Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T19:38:45.021290Z info FLAG: --log_target="[stdout]" 2026-11-03T19:38:45.021293Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T19:38:45.021295Z info FLAG: --outlierLogPath="" 2026-11-03T19:38:45.021297Z info FLAG: --profiling="true" 2026-11-03T19:38:45.021299Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T19:38:45.021301Z info FLAG: --proxyLogLevel="warning" 2026-11-03T19:38:45.021303Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T19:38:45.021305Z info FLAG: --stsPort="0" 2026-11-03T19:38:45.021306Z info FLAG: --templateFile="" 2026-11-03T19:38:45.021308Z info FLAG: --tokenManagerPlugin="" 2026-11-03T19:38:45.021311Z info FLAG: --vklog="0" 2026-11-03T19:38:45.021313Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T19:38:45.021508Z info Proxy role ips=[10.132.0.165] type=router id=maas-default-gateway-openshift-default-77f556d647-bkdvr.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T19:38:45.021559Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T19:38:45.022788Z info cpu limit detected as 2, setting concurrency 2026-11-03T19:38:45.023058Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T19:38:45.023069Z info JWT policy is third-party-jwt 2026-11-03T19:38:45.023072Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T19:38:45.024893Z info Starting default Istio SDS Server 2026-11-03T19:38:45.024913Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T19:38:45.024943Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T19:38:45.025219Z info Opening status port 15020 2026-11-03T19:38:45.026159Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T19:38:45.027526Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T19:38:45.028437Z info Starting proxy agent 2026-11-03T19:38:45.028468Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T19:38:45.046264Z info sds Starting SDS grpc server 2026-11-03T19:38:45.046285Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T19:38:45.090513Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T19:38:45.176926Z info cache generated new workload certificate resourceName=default latency=150.971459ms ttl=23h59m59.823078708s 2026-11-03T19:38:45.176962Z info cache Root cert has changed, start rotating root cert 2026-11-03T19:38:45.176992Z info cache returned workload trust anchor from cache ttl=23h59m59.823008851s 2026-11-03T19:38:45.238379Z info ads ADS: new connection for node:1 2026-11-03T19:38:45.238450Z info cache returned workload certificate from cache ttl=23h59m59.761552833s 2026-11-03T19:38:45.238670Z info ads ADS: new connection for node:2 2026-11-03T19:38:45.238736Z info cache returned workload trust anchor from cache ttl=23h59m59.761265773s 2026-11-03T19:38:45.335562Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:38:53.288090Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:38:54.232232Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T19:38:54.518452Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T19:38:55.374048Z info Readiness succeeded in 10.355869709s 2026-11-03T19:38:55.374406Z info Envoy proxy is ready 2026-11-03T19:39:31.432030Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:39:37.300750Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T19:39:37.300788Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T19:39:37.300803Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T19:39:37.300807Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T19:39:37.171Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 87 329 129 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "0e160230-bddc-4296-a76d-3ef9560a90bc" "maas.apps.9a15ddbd156184f738d5.prod.konflux-ocp-ci.dev" "10.132.0.53:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.165:59754 10.132.0.165:443 10.132.0.12:57798 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T19:39:37.912Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 78 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "be5f0fbb-d569-44f8-acf1-7ee8c1bfbcfa" "maas.apps.9a15ddbd156184f738d5.prod.konflux-ocp-ci.dev" "10.132.0.56:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.165:34968 10.132.0.165:443 10.132.0.12:57806 - v1-completions-path 2026-11-03T19:39:53.311258Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:39:57.986767Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:40:24.649272Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:40:29.304266Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:40:34.253052Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:40:42.998493Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:40:48.265515Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T19:40:48.265551Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T19:40:48.265564Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T19:40:48.265569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T19:40:48.154Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 77 319 111 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "217eb3d0-c2f1-489c-94e4-fc7ec70584b9" "maas.apps.9a15ddbd156184f738d5.prod.konflux-ocp-ci.dev" "10.132.0.53:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.165:54700 10.132.0.165:443 10.132.0.12:40144 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T19:40:50.614Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 298 82 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "6be6cb0b-39f3-4d1a-8127-f67dc8316935" "maas.apps.9a15ddbd156184f738d5.prod.konflux-ocp-ci.dev" "10.132.0.56:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.165:45006 10.132.0.165:443 10.132.0.12:40152 - v1-completions-path [2026-11-03T19:40:53.064Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 60 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "1b28aa4c-eaa2-4875-b958-ed888edac902" "maas.apps.9a15ddbd156184f738d5.prod.konflux-ocp-ci.dev" "10.132.0.56:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.165:45006 10.132.0.165:443 10.132.0.12:51886 - v1-completions-path 2026-11-03T19:40:54.131651Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-03T19:40:54.127Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 62 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "8b5989bb-f87f-4c8b-a508-ccae2b6098ee" "maas.apps.9a15ddbd156184f738d5.prod.konflux-ocp-ci.dev" "10.132.0.56:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.165:45006 10.132.0.165:443 10.132.0.12:51898 - v1-completions-path 2026-11-03T19:41:12.479766Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:41:23.254426Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:41:29.732658Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:41:40.627676Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T19:41:40.627711Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T19:41:40.627725Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T19:41:40.627730Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T19:41:40.512Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 317 115 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "52079f9a-eb8c-4351-9441-38fe663f144c" "maas.apps.9a15ddbd156184f738d5.prod.konflux-ocp-ci.dev" "10.132.0.53:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.165:38876 10.132.0.165:443 10.132.0.12:58378 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T19:41:58.581494Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:42:03.252013Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:42:26.817222Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:42:33.300252Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:42:59.069170Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:43:17.736586Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:47:50.203630Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:48:07.671948Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:48:12.311123Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:48:29.202238Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:50:12.121175Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T19:50:12.121212Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T19:50:12.121227Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T19:50:12.121232Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T19:50:12.009Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 568 111 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "3892b93f-8670-44b9-a760-77b4a2031b72" "maas.apps.9a15ddbd156184f738d5.prod.konflux-ocp-ci.dev" "10.132.0.53:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.165:44538 10.132.0.165:443 10.132.0.12:39214 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T19:50:22.502650Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:51:04.228384Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:51:31.070145Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:51:46.487033Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:52:02.442798Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:52:14.902053Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:52:19.816774Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:52:48.692487Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:52:59.504522Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:53:10.694816Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:53:17.474044Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-03T19:53:17.474300Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-11-03T19:53:17.397Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 76 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "52d47327-75a8-45d5-a9c2-7851b15aa092" "maas.apps.9a15ddbd156184f738d5.prod.konflux-ocp-ci.dev" "10.132.0.55:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.165:58844 10.132.0.165:443 10.132.0.12:43540 - v1-catch-all-path 2026-11-03T19:53:33.602763Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:53:53.817093Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T19:54:05.093626Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8