2026-11-04T05:40:54.403578Z info FLAG: --concurrency="0" 2026-11-04T05:40:54.403598Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-04T05:40:54.403604Z info FLAG: --help="false" 2026-11-04T05:40:54.403607Z info FLAG: --log_as_json="false" 2026-11-04T05:40:54.403611Z info FLAG: --log_caller="" 2026-11-04T05:40:54.403614Z info FLAG: --log_output_level="default:info" 2026-11-04T05:40:54.403617Z info FLAG: --log_stacktrace_level="default:none" 2026-11-04T05:40:54.403623Z info FLAG: --log_target="[stdout]" 2026-11-04T05:40:54.403627Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-04T05:40:54.403629Z info FLAG: --outlierLogPath="" 2026-11-04T05:40:54.403633Z info FLAG: --profiling="true" 2026-11-04T05:40:54.403636Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-04T05:40:54.403639Z info FLAG: --proxyLogLevel="warning" 2026-11-04T05:40:54.403642Z info FLAG: --serviceCluster="istio-proxy" 2026-11-04T05:40:54.403645Z info FLAG: --stsPort="0" 2026-11-04T05:40:54.403648Z info FLAG: --templateFile="" 2026-11-04T05:40:54.403651Z info FLAG: --tokenManagerPlugin="" 2026-11-04T05:40:54.403655Z info FLAG: --vklog="0" 2026-11-04T05:40:54.403659Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-04T05:40:54.403856Z info Proxy role ips=[10.133.0.30] type=router id=maas-default-gateway-openshift-default-694db55c87-6x6f8.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-04T05:40:54.403926Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-04T05:40:54.405588Z info cpu limit detected as 2, setting concurrency 2026-11-04T05:40:54.406413Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-04T05:40:54.406431Z info JWT policy is third-party-jwt 2026-11-04T05:40:54.406436Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-04T05:40:54.408314Z info Starting default Istio SDS Server 2026-11-04T05:40:54.408353Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-04T05:40:54.408379Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-04T05:40:54.408873Z info Opening status port 15020 2026-11-04T05:40:54.410442Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-04T05:40:54.412265Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-04T05:40:54.413459Z info Starting proxy agent 2026-11-04T05:40:54.413499Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-04T05:40:54.426388Z info sds Starting SDS grpc server 2026-11-04T05:40:54.426463Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-04T05:40:54.494381Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-04T05:40:54.603772Z info ads ADS: new connection for node:1 2026-11-04T05:40:54.604182Z info ads ADS: new connection for node:2 2026-11-04T05:40:54.651643Z info cache generated new workload certificate resourceName=default latency=241.019533ms ttl=23h59m59.348360344s 2026-11-04T05:40:54.651687Z info cache Root cert has changed, start rotating root cert 2026-11-04T05:40:54.651727Z info cache returned workload trust anchor from cache ttl=23h59m59.348274074s 2026-11-04T05:40:54.651742Z info cache returned workload certificate from cache ttl=23h59m59.348258499s 2026-11-04T05:40:54.651845Z info cache returned workload trust anchor from cache ttl=23h59m59.34815835s 2026-11-04T05:40:54.652182Z info cache returned workload trust anchor from cache ttl=23h59m59.347818892s 2026-11-04T05:40:54.750729Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T05:41:03.026122Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T05:41:04.059688Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-04T05:41:05.793206Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T05:41:05.825040Z info Readiness succeeded in 11.425231322s 2026-11-04T05:41:05.825387Z info Envoy proxy is ready [2026-11-04T05:41:57.068Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 64 - "34.228.250.194,10.132.0.9" "curl/7.76.1" "9bf40f8c-789c-4291-97fb-91fc415e1db6" "maas.apps.853140fcaf2a1a13ab9e.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.30:443 10.132.0.9:56530 - v1-model-routing 2026-11-04T05:41:58.225860Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T05:41:58.225895Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T05:41:58.225908Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T05:41:58.225913Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-04T05:41:58.206Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 91 19 - "34.228.250.194,10.132.0.9" "curl/7.76.1" "1c09b9e4-298a-4367-913c-636b6869b743" "maas.apps.853140fcaf2a1a13ab9e.prod.konflux-ocp-ci.dev" "10.134.0.39:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.30:35508 10.133.0.30:443 10.132.0.9:56534 - odh-ai-gateway-infra.maas-api-route.4 2026-11-04T05:43:58.846773Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T05:43:58.846804Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T05:43:58.846816Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T05:43:58.846820Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-04T05:43:58.709Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 137 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "f1b82fd6-96e6-49b3-9201-e95f5e776853" "maas.apps.853140fcaf2a1a13ab9e.prod.konflux-ocp-ci.dev" "10.134.0.39:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.30:46770 10.133.0.30:443 10.132.0.9:40210 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T05:44:00.376493Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T05:44:00.376550Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T05:44:00.376563Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T05:44:00.376569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T05:44:00.250Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 126 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "860c9895-44e7-4b8d-9c84-9ae6d9ff1008" "maas.apps.853140fcaf2a1a13ab9e.prod.konflux-ocp-ci.dev" "10.134.0.39:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.30:46776 10.133.0.30:443 10.132.0.9:40224 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T05:44:07.528Z] "POST /maas-api/internal/v1/api-keys/cleanup HTTP/1.1" 404 NR route_not_found - "-" 2 0 11 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "bc094b82-f8dd-4b04-9374-ce37138c4af1" "maas.apps.853140fcaf2a1a13ab9e.prod.konflux-ocp-ci.dev" "-" - - 10.133.0.30:443 10.132.0.9:58768 - - 2026-11-04T05:44:08.139536Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T05:44:08.139659Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T05:44:08.139706Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T05:44:08.139728Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T05:44:08.074Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 64 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "eb718274-6327-4c19-9b2c-6f31306926da" "maas.apps.853140fcaf2a1a13ab9e.prod.konflux-ocp-ci.dev" "10.134.0.39:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.30:46776 10.133.0.30:443 10.132.0.9:58774 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T05:44:13.835Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 59 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "bdbfb00d-9622-41d5-9ce9-65f8c6bf1e7a" "maas.apps.853140fcaf2a1a13ab9e.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.30:443 10.132.0.9:49646 - v1-completions-path 2026-11-04T05:45:10.468989Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T05:45:10.469036Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T05:45:10.469058Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T05:45:10.469066Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T05:45:10.323Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 145 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "351ae5a4-fd5c-40bd-810d-232ac7a99894" "maas.apps.853140fcaf2a1a13ab9e.prod.konflux-ocp-ci.dev" "10.134.0.39:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.30:37622 10.133.0.30:443 10.132.0.9:50416 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T05:45:10.957911Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T05:45:10.957945Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T05:45:10.957957Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T05:45:10.957962Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-04T05:45:10.531Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 86 328 426 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "1428e09c-d7de-46c6-a88a-42967c039212" "maas.apps.853140fcaf2a1a13ab9e.prod.konflux-ocp-ci.dev" "10.134.0.39:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.30:37622 10.133.0.30:443 10.132.0.9:50432 - odh-ai-gateway-infra.maas-api-route.3