2026-11-04T03:57:42.881932Z info FLAG: --concurrency="0" 2026-11-04T03:57:42.881951Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-04T03:57:42.881955Z info FLAG: --help="false" 2026-11-04T03:57:42.881958Z info FLAG: --log_as_json="false" 2026-11-04T03:57:42.881960Z info FLAG: --log_caller="" 2026-11-04T03:57:42.881962Z info FLAG: --log_output_level="default:info" 2026-11-04T03:57:42.881964Z info FLAG: --log_stacktrace_level="default:none" 2026-11-04T03:57:42.881968Z info FLAG: --log_target="[stdout]" 2026-11-04T03:57:42.881970Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-04T03:57:42.881972Z info FLAG: --outlierLogPath="" 2026-11-04T03:57:42.881974Z info FLAG: --profiling="true" 2026-11-04T03:57:42.881976Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-04T03:57:42.881978Z info FLAG: --proxyLogLevel="warning" 2026-11-04T03:57:42.881981Z info FLAG: --serviceCluster="istio-proxy" 2026-11-04T03:57:42.881983Z info FLAG: --stsPort="0" 2026-11-04T03:57:42.881985Z info FLAG: --templateFile="" 2026-11-04T03:57:42.881987Z info FLAG: --tokenManagerPlugin="" 2026-11-04T03:57:42.881990Z info FLAG: --vklog="0" 2026-11-04T03:57:42.881993Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-04T03:57:42.882148Z info Proxy role ips=[10.132.0.57] type=router id=maas-default-gateway-openshift-default-7d64fcddb7-n5x92.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-04T03:57:42.882198Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-04T03:57:42.883206Z info cpu limit detected as 2, setting concurrency 2026-11-04T03:57:42.883413Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-04T03:57:42.883424Z info JWT policy is third-party-jwt 2026-11-04T03:57:42.883427Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-04T03:57:42.885467Z info Starting default Istio SDS Server 2026-11-04T03:57:42.885496Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-04T03:57:42.885521Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-04T03:57:42.885727Z info Opening status port 15020 2026-11-04T03:57:42.886979Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-04T03:57:42.889016Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-04T03:57:42.890323Z info Starting proxy agent 2026-11-04T03:57:42.890369Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-04T03:57:42.898111Z info sds Starting SDS grpc server 2026-11-04T03:57:42.898139Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-04T03:57:42.956442Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-04T03:57:42.992634Z info cache generated new workload certificate resourceName=default latency=106.158061ms ttl=23h59m59.007378729s 2026-11-04T03:57:42.992684Z info cache Root cert has changed, start rotating root cert 2026-11-04T03:57:42.992726Z info cache returned workload trust anchor from cache ttl=23h59m59.007274838s 2026-11-04T03:57:43.111123Z info ads ADS: new connection for node:1 2026-11-04T03:57:43.111258Z info cache returned workload certificate from cache ttl=23h59m58.888744879s 2026-11-04T03:57:43.112660Z info ads ADS: new connection for node:2 2026-11-04T03:57:43.112725Z info cache returned workload trust anchor from cache ttl=23h59m58.887275629s 2026-11-04T03:57:43.203406Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-04T03:57:51.524459Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-04T03:57:52.460445Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-04T03:57:53.763653Z info Readiness succeeded in 10.884084198s 2026-11-04T03:57:53.764484Z info Envoy proxy is ready 2026-11-04T03:58:19.631897Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T03:58:19.631958Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T03:58:19.631972Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T03:58:19.631977Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-04T03:58:19.561Z] "DELETE /v1/api-keys/d4498fe1-d200-46b0-8fb8-5558a1899fdb HTTP/1.1" 200 - via_upstream - "-" 0 365 70 - "44.212.242.249,10.132.0.9" "curl/7.76.1" "2a75617d-7baf-4d22-b6a9-e24870ef8795" "maas.apps.9a8581fc44250847a02c.prod.konflux-ocp-ci.dev" "10.132.0.50:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.57:39352 10.132.0.57:443 10.132.0.9:34718 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T03:58:19.725608Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T03:58:19.725639Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T03:58:19.725651Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T03:58:19.725656Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-04T03:58:19.710Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 91 14 - "44.212.242.249,10.132.0.9" "curl/7.76.1" "7266b64e-5450-4914-a684-361c024f3a6d" "maas.apps.9a8581fc44250847a02c.prod.konflux-ocp-ci.dev" "10.132.0.50:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.57:39352 10.132.0.57:443 10.132.0.9:34726 - odh-ai-gateway-infra.maas-api-route.4 2026-11-04T04:00:09.442003Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T04:00:09.442035Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T04:00:09.442047Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T04:00:09.442053Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T04:00:09.376Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 66 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "948b0fe6-dbc5-49cd-81e3-989837c0cb77" "maas.apps.9a8581fc44250847a02c.prod.konflux-ocp-ci.dev" "10.132.0.50:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.57:54886 10.132.0.57:443 10.132.0.9:42182 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T04:00:09.893521Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T04:00:09.893559Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T04:00:09.893576Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T04:00:09.893583Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T04:00:10.295438Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T04:00:10.295469Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T04:00:10.295482Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T04:00:10.295486Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-04T04:00:09.822Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 71 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "6f7781fe-7d64-4c98-8474-b81230c6eaf6" "maas.apps.9a8581fc44250847a02c.prod.konflux-ocp-ci.dev" "10.132.0.50:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.57:54886 10.132.0.57:443 10.132.0.9:42188 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T04:00:10.236Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 58 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "774ab76a-4244-4fd3-8e2d-3ea677502732" "maas.apps.9a8581fc44250847a02c.prod.konflux-ocp-ci.dev" "10.132.0.50:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.57:54886 10.132.0.57:443 10.132.0.9:42200 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T04:00:26.124475Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T04:00:26.124509Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T04:00:26.124526Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T04:00:26.124533Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T04:00:26.061Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 63 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "805c3004-0972-4a47-b7c1-ce5c88803b45" "maas.apps.9a8581fc44250847a02c.prod.konflux-ocp-ci.dev" "10.132.0.50:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.57:54886 10.132.0.57:443 10.132.0.9:45512 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T04:00:40.535331Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T04:00:40.535365Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T04:00:40.535373Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-11-04T04:00:40.535385Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T04:00:40.535390Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T04:00:40.700396Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T04:00:40.700428Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T04:00:40.700440Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T04:00:40.700445Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-04T04:00:40.451Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 83 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "fa34fc7d-845d-4f4c-b9c6-b73fd5e644d3" "maas.apps.9a8581fc44250847a02c.prod.konflux-ocp-ci.dev" "10.132.0.50:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.57:54886 10.132.0.57:443 10.132.0.9:51988 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T04:00:40.592Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 86 329 108 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "71269e37-6818-4c48-8454-700d765ec571" "maas.apps.9a8581fc44250847a02c.prod.konflux-ocp-ci.dev" "10.132.0.50:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.57:54086 10.132.0.57:443 10.132.0.9:51996 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T04:01:24.067825Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T04:01:24.067982Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T04:01:24.068053Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T04:01:24.068084Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T04:01:24.068401Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 2026-11-04T04:01:24.068510Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-04T04:01:24.068542Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-04T04:01:24.068566Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-04T04:01:24.068590Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-11-04T04:01:23.945Z] "POST /v1/completions HTTP/1.1" 404 - via_upstream - "-" 94 126 123 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "45ae3902-62d6-4b76-b64e-34f044f9458c" "maas.apps.9a8581fc44250847a02c.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.57:49752 10.132.0.57:443 10.132.0.9:40754 - v1-completions-path