2026-10-30T13:22:21.797107Z info FLAG: --concurrency="0" 2026-10-30T13:22:21.797131Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T13:22:21.797136Z info FLAG: --help="false" 2026-10-30T13:22:21.797140Z info FLAG: --log_as_json="false" 2026-10-30T13:22:21.797143Z info FLAG: --log_caller="" 2026-10-30T13:22:21.797146Z info FLAG: --log_output_level="default:info" 2026-10-30T13:22:21.797150Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T13:22:21.797156Z info FLAG: --log_target="[stdout]" 2026-10-30T13:22:21.797160Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T13:22:21.797163Z info FLAG: --outlierLogPath="" 2026-10-30T13:22:21.797166Z info FLAG: --profiling="true" 2026-10-30T13:22:21.797169Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T13:22:21.797173Z info FLAG: --proxyLogLevel="warning" 2026-10-30T13:22:21.797176Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T13:22:21.797179Z info FLAG: --stsPort="0" 2026-10-30T13:22:21.797182Z info FLAG: --templateFile="" 2026-10-30T13:22:21.797185Z info FLAG: --tokenManagerPlugin="" 2026-10-30T13:22:21.797189Z info FLAG: --vklog="0" 2026-10-30T13:22:21.797193Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T13:22:21.797389Z info Proxy role ips=[10.134.0.117] type=router id=maas-default-gateway-openshift-default-c9ffffc7f-pmfxx.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T13:22:21.797444Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T13:22:21.799166Z info cpu limit detected as 2, setting concurrency 2026-10-30T13:22:21.799524Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T13:22:21.799540Z info JWT policy is third-party-jwt 2026-10-30T13:22:21.799543Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T13:22:21.801348Z info Starting default Istio SDS Server 2026-10-30T13:22:21.801374Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T13:22:21.801397Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T13:22:21.802326Z info Opening status port 15020 2026-10-30T13:22:21.802780Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T13:22:21.803390Z info sds Starting SDS grpc server 2026-10-30T13:22:21.803409Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T13:22:21.804551Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T13:22:21.805423Z info Starting proxy agent 2026-10-30T13:22:21.805506Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T13:22:21.888920Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T13:22:22.010334Z info ads ADS: new connection for node:1 2026-10-30T13:22:22.010810Z info ads ADS: new connection for node:2 2026-10-30T13:22:22.067764Z info cache generated new workload certificate resourceName=default latency=264.266823ms ttl=23h59m59.932241247s 2026-10-30T13:22:22.067800Z info cache Root cert has changed, start rotating root cert 2026-10-30T13:22:22.067833Z info cache returned workload trust anchor from cache ttl=23h59m59.932167051s 2026-10-30T13:22:22.067854Z info cache returned workload certificate from cache ttl=23h59m59.932146645s 2026-10-30T13:22:22.067945Z info cache returned workload trust anchor from cache ttl=23h59m59.932058974s 2026-10-30T13:22:22.068191Z info cache returned workload trust anchor from cache ttl=23h59m59.931809366s 2026-10-30T13:22:22.158024Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:22:30.355625Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:22:31.073670Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-10-30T13:22:31.364854Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-10-30T13:22:32.899240Z info Readiness succeeded in 11.105028546s 2026-10-30T13:22:32.899613Z info Envoy proxy is ready 2026-10-30T13:23:30.916970Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T13:23:35.538Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 81 - "52.71.36.72,10.132.0.22" "python-requests/2.34.2" "1ba7f050-383d-4e3b-a229-55019bd711b8" "maas.apps.03ade695efcb3fb1e0e7.prod.konflux-ocp-ci.dev" "10.132.0.48:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.117:60774 10.134.0.117:443 10.132.0.22:39972 - v1-completions-path 2026-10-30T13:23:47.948261Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:23:52.412071Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T13:23:55.746Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 59 - "52.71.36.72,10.132.0.22" "python-requests/2.34.2" "2a12d586-374b-40f4-a146-d8e3081905cb" "maas.apps.03ade695efcb3fb1e0e7.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.117:443 10.132.0.22:42562 - v1-completions-path 2026-10-30T13:24:07.994023Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:24:33.744788Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T13:24:40.797Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 59 - "52.71.36.72,10.132.0.22" "python-requests/2.34.2" "b267ca0f-59c2-4f69-ad52-099e4bbf7ac2" "maas.apps.03ade695efcb3fb1e0e7.prod.konflux-ocp-ci.dev" "10.132.0.48:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.117:60774 10.134.0.117:443 10.132.0.22:55312 - v1-completions-path 2026-10-30T13:24:57.149799Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:25:07.497730Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:25:11.915250Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:25:20.636179Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T13:25:20.636218Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T13:25:20.636236Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T13:25:20.636242Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T13:25:20.553Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 317 83 - "52.71.36.72,10.132.0.22" "python-requests/2.34.2" "960b8fd8-a3cd-4dfb-9885-179a22e97281" "maas.apps.03ade695efcb3fb1e0e7.prod.konflux-ocp-ci.dev" "10.133.0.35:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.117:54280 10.134.0.117:443 10.132.0.22:41938 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T13:25:21.314861Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-10-30T13:25:21.310Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 57 - "52.71.36.72,10.132.0.22" "python-requests/2.34.2" "2ed76cad-8a2b-45a3-9c68-f8a7b7ae1016" "maas.apps.03ade695efcb3fb1e0e7.prod.konflux-ocp-ci.dev" "10.132.0.48:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.117:60774 10.134.0.117:443 10.132.0.22:41948 - v1-completions-path [2026-10-30T13:25:21.552Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 56 - "52.71.36.72,10.132.0.22" "python-requests/2.34.2" "fa2b8588-8ff1-460f-9a99-978436430b92" "maas.apps.03ade695efcb3fb1e0e7.prod.konflux-ocp-ci.dev" "10.132.0.48:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.117:60774 10.134.0.117:443 10.132.0.22:58942 - v1-completions-path 2026-10-30T13:25:33.854411Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:25:35.856951Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-10-30T13:25:36.253692Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:25:55.420417Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:25:59.830994Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:26:16.488910Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:26:33.641884Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:26:45.447353Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:26:48.835967Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:27:06.800322Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:27:09.688313Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:27:34.439863Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:27:37.171872Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:29:34.873123Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:29:37.418831Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:31:19.338470Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:31:34.389865Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:31:36.710131Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:31:51.301110Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:33:59.828032Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T13:33:59.828075Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T13:33:59.828093Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T13:33:59.828100Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T13:33:59.736Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 33 325 92 - "52.71.36.72,10.132.0.22" "python-requests/2.34.2" "adf8a447-7012-42e1-903a-c4925e149d93" "maas.apps.03ade695efcb3fb1e0e7.prod.konflux-ocp-ci.dev" "10.133.0.35:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.117:38470 10.134.0.117:443 10.132.0.22:44464 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T13:34:04.343559Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:34:21.459495Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:34:35.646934Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T13:34:35.646972Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T13:34:35.646989Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T13:34:35.646994Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T13:34:35.593Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 630 53 - "52.71.36.72,10.132.0.22" "python-requests/2.34.2" "3a08cb07-6060-4734-8b02-eef7f9426465" "maas.apps.03ade695efcb3fb1e0e7.prod.konflux-ocp-ci.dev" "10.133.0.35:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.117:38470 10.134.0.117:443 10.132.0.22:44438 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T13:34:37.759664Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:34:49.718647Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:34:53.664995Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:35:05.084169Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:35:18.579081Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:35:35.567541Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:35:40.449552Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:35:58.653385Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:36:04.235204Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-10-30T13:36:04.149Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 85 - "52.71.36.72,10.132.0.22" "python-requests/2.34.2" "620e02c1-196c-4eea-8b49-c5eb0630b9e9" "maas.apps.03ade695efcb3fb1e0e7.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.117:52838 10.134.0.117:443 10.132.0.22:44794 - v1-catch-all-path 2026-10-30T13:36:16.693705Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:36:19.038928Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:36:33.843416Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:36:44.797544Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8