2026-11-03T22:46:10.625154Z info FLAG: --concurrency="0" 2026-11-03T22:46:10.625176Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T22:46:10.625180Z info FLAG: --help="false" 2026-11-03T22:46:10.625183Z info FLAG: --log_as_json="false" 2026-11-03T22:46:10.625186Z info FLAG: --log_caller="" 2026-11-03T22:46:10.625189Z info FLAG: --log_output_level="default:info" 2026-11-03T22:46:10.625192Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T22:46:10.625198Z info FLAG: --log_target="[stdout]" 2026-11-03T22:46:10.625201Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T22:46:10.625204Z info FLAG: --outlierLogPath="" 2026-11-03T22:46:10.625207Z info FLAG: --profiling="true" 2026-11-03T22:46:10.625210Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T22:46:10.625213Z info FLAG: --proxyLogLevel="warning" 2026-11-03T22:46:10.625217Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T22:46:10.625220Z info FLAG: --stsPort="0" 2026-11-03T22:46:10.625223Z info FLAG: --templateFile="" 2026-11-03T22:46:10.625225Z info FLAG: --tokenManagerPlugin="" 2026-11-03T22:46:10.625229Z info FLAG: --vklog="0" 2026-11-03T22:46:10.625232Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T22:46:10.625430Z info Proxy role ips=[10.132.0.162] type=router id=maas-default-gateway-openshift-default-7f5667f86c-zhbvw.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T22:46:10.625496Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T22:46:10.626617Z info cpu limit detected as 2, setting concurrency 2026-11-03T22:46:10.626836Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T22:46:10.626852Z info JWT policy is third-party-jwt 2026-11-03T22:46:10.626856Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T22:46:10.630848Z info Starting default Istio SDS Server 2026-11-03T22:46:10.630873Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T22:46:10.630896Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T22:46:10.631212Z info Opening status port 15020 2026-11-03T22:46:10.631720Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T22:46:10.633149Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T22:46:10.634113Z info Starting proxy agent 2026-11-03T22:46:10.634178Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T22:46:10.650829Z info sds Starting SDS grpc server 2026-11-03T22:46:10.650855Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T22:46:10.727622Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T22:46:10.788655Z info cache generated new workload certificate resourceName=default latency=157.128635ms ttl=23h59m59.211349517s 2026-11-03T22:46:10.788780Z info cache Root cert has changed, start rotating root cert 2026-11-03T22:46:10.788830Z info cache returned workload trust anchor from cache ttl=23h59m59.211170532s 2026-11-03T22:46:11.087484Z info ads ADS: new connection for node:1 2026-11-03T22:46:11.087690Z info cache returned workload certificate from cache ttl=23h59m58.912312569s 2026-11-03T22:46:11.090853Z info ads ADS: new connection for node:2 2026-11-03T22:46:11.091087Z info cache returned workload trust anchor from cache ttl=23h59m58.90891689s 2026-11-03T22:46:11.265950Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:46:21.404579Z info Readiness succeeded in 10.781935305s 2026-11-03T22:46:21.404916Z info Envoy proxy is ready 2026-11-03T22:46:59.931872Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:47:25.295137Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:55:06.800259Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:55:34.576855Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:57:09.348207Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:02:39.784418Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:03:19.921544Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:08:49.229066Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:09:15.826384Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:10:11.666296Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:10:17.341606Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:10:17.341649Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:10:17.341666Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T23:10:17.341673Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T23:10:17.218Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 87 329 122 - "34.228.250.194,10.132.0.14" "python-requests/2.34.2" "d4d0bba2-d2de-4b16-8602-f110bd7bb65d" "maas.apps.026456f0543d81eedc25.prod.konflux-ocp-ci.dev" "10.134.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.162:52828 10.132.0.162:443 10.132.0.14:44710 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T23:10:17.554Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 82 - "34.228.250.194,10.132.0.14" "python-requests/2.34.2" "c3e59e55-b06c-4253-b1d8-d22043974093" "maas.apps.026456f0543d81eedc25.prod.konflux-ocp-ci.dev" "10.134.0.35:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.162:46594 10.132.0.162:443 10.132.0.14:44712 - v1-completions-path 2026-11-03T23:10:33.187537Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:10:37.916264Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:11:04.477486Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:11:21.329498Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T23:11:34.645Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 69 - "34.228.250.194,10.132.0.14" "python-requests/2.34.2" "fa809256-9509-95a8-b4c9-1bb8fddbfdb4" "maas.apps.026456f0543d81eedc25.prod.konflux-ocp-ci.dev" "10.134.0.35:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.162:35790 10.132.0.162:443 10.132.0.14:43298 - v1-completions-path 2026-11-03T23:11:52.614664Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:12:03.140063Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:12:09.601655Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T23:12:24.388Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 60 - "34.228.250.194,10.132.0.14" "python-requests/2.34.2" "04280af2-5bb5-4c37-b874-21652497d3e4" "maas.apps.026456f0543d81eedc25.prod.konflux-ocp-ci.dev" "10.134.0.35:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.162:35790 10.132.0.162:443 10.132.0.14:43942 - v1-completions-path 2026-11-03T23:12:37.945901Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:12:42.589195Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:13:05.919464Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:13:12.339622Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:13:38.300523Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:14:00.396150Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:14:13.403378Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:14:19.317016Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:14:43.954730Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:14:49.385377Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:15:09.608688Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:15:32.595072Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:15:35.657348Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-03T23:15:43.294734Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:16:01.656320Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:16:07.530406Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:18:31.725185Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:18:37.438056Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:20:18.703723Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:20:35.880231Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:20:40.478615Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:21:04.799197Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:23:12.297889Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:23:54.114649Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:24:23.647120Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T23:24:23.647155Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T23:24:23.647167Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-03T23:24:23.647173Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 [2026-11-03T23:24:23.533Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 80 322 113 - "34.228.250.194,10.132.0.14" "python-requests/2.34.2" "c7cee84e-623f-427d-9ad4-dcd964487919" "maas.apps.026456f0543d81eedc25.prod.konflux-ocp-ci.dev" "10.134.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.162:50320 10.132.0.162:443 10.132.0.14:42316 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T23:24:26.198107Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:24:41.429041Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:24:53.653130Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:24:59.020058Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:25:08.884111Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:25:13.506075Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:25:36.802356Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:29:09.834112Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:29:22.695359Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:29:28.682256Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:29:28.682290Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:29:28.682302Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T23:29:28.682307Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:29:28.960806Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-03T23:29:28.962440Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-11-03T23:29:28.567Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 318 115 - "34.228.250.194,10.132.0.14" "python-requests/2.34.2" "66c8a681-9ab2-4dec-bf5c-744cb6d5ffdc" "maas.apps.026456f0543d81eedc25.prod.konflux-ocp-ci.dev" "10.133.0.134:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.162:34872 10.132.0.162:443 10.132.0.14:58216 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T23:29:28.884Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 78 - "34.228.250.194,10.132.0.14" "python-requests/2.34.2" "f1725268-4b8a-408a-8007-9d163b595e18" "maas.apps.026456f0543d81eedc25.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.162:33968 10.132.0.162:443 10.132.0.14:58218 - v1-catch-all-path [2026-11-03T23:29:30.033Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 429 - - "-" 61 18 58 - "34.228.250.194,10.132.0.14" "python-requests/2.34.2" "bb6c528c-e86f-4845-a4a9-eee8f8261e00" "maas.apps.026456f0543d81eedc25.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.162:443 10.132.0.14:58228 - v1-catch-all-path 2026-11-03T23:29:43.714390Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:29:48.468823Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:30:02.992345Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:30:14.533590Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8