2026-11-04T08:53:25.265491Z info FLAG: --concurrency="0" 2026-11-04T08:53:25.265510Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-04T08:53:25.265514Z info FLAG: --help="false" 2026-11-04T08:53:25.265516Z info FLAG: --log_as_json="false" 2026-11-04T08:53:25.265519Z info FLAG: --log_caller="" 2026-11-04T08:53:25.265521Z info FLAG: --log_output_level="default:info" 2026-11-04T08:53:25.265523Z info FLAG: --log_stacktrace_level="default:none" 2026-11-04T08:53:25.265527Z info FLAG: --log_target="[stdout]" 2026-11-04T08:53:25.265529Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-04T08:53:25.265531Z info FLAG: --outlierLogPath="" 2026-11-04T08:53:25.265533Z info FLAG: --profiling="true" 2026-11-04T08:53:25.265536Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-04T08:53:25.265538Z info FLAG: --proxyLogLevel="warning" 2026-11-04T08:53:25.265540Z info FLAG: --serviceCluster="istio-proxy" 2026-11-04T08:53:25.265542Z info FLAG: --stsPort="0" 2026-11-04T08:53:25.265544Z info FLAG: --templateFile="" 2026-11-04T08:53:25.265546Z info FLAG: --tokenManagerPlugin="" 2026-11-04T08:53:25.265549Z info FLAG: --vklog="0" 2026-11-04T08:53:25.265551Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-04T08:53:25.265765Z info Proxy role ips=[10.134.0.37] type=router id=maas-default-gateway-openshift-default-855f9574cf-fcq6m.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-04T08:53:25.265819Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-04T08:53:25.266805Z info cpu limit detected as 2, setting concurrency 2026-11-04T08:53:25.266997Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-04T08:53:25.267007Z info JWT policy is third-party-jwt 2026-11-04T08:53:25.267010Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-04T08:53:25.268540Z info Starting default Istio SDS Server 2026-11-04T08:53:25.268560Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-04T08:53:25.268578Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-04T08:53:25.269032Z info Opening status port 15020 2026-11-04T08:53:25.269916Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-04T08:53:25.271237Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-04T08:53:25.272101Z info Starting proxy agent 2026-11-04T08:53:25.272132Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-04T08:53:25.281619Z info sds Starting SDS grpc server 2026-11-04T08:53:25.281636Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-04T08:53:25.334754Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-04T08:53:25.425064Z info cache generated new workload certificate resourceName=default latency=155.389888ms ttl=23h59m59.574941362s 2026-11-04T08:53:25.425189Z info cache Root cert has changed, start rotating root cert 2026-11-04T08:53:25.425235Z info cache returned workload trust anchor from cache ttl=23h59m59.574765775s 2026-11-04T08:53:25.440410Z info ads ADS: new connection for node:1 2026-11-04T08:53:25.440603Z info cache returned workload certificate from cache ttl=23h59m59.559536173s 2026-11-04T08:53:25.441475Z info ads ADS: new connection for node:2 2026-11-04T08:53:25.441598Z info cache returned workload trust anchor from cache ttl=23h59m59.558404347s 2026-11-04T08:53:25.487118Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:53:31.879772Z info Readiness succeeded in 6.617043219s 2026-11-04T08:53:31.880074Z info Envoy proxy is ready 2026-11-04T08:53:53.363040Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:53:58.149112Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:54:01.774404Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:54:06.322745Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:54:09.337380Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:54:12.828321Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:54:16.937315Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T08:54:42.439Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 115 - "34.228.250.194,10.132.0.16" "curl/7.76.1" "e594d86f-52c7-4450-b22f-767271d7629e" "maas.apps.a3dc5cee932be05f4d88.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.37:443 10.132.0.16:36912 - v1-model-routing 2026-11-04T08:55:00.653559Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-04T08:55:00.521Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 132 - "34.228.250.194,10.132.0.16" "python-requests/2.33.0" "74531e74-8991-4a4c-b490-6d79908357ee" "maas.apps.a3dc5cee932be05f4d88.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.37:54078 10.134.0.37:443 10.132.0.16:49706 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T08:55:01.713908Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T08:55:01.576Z] "DELETE /v1/api-keys/4a114efb-d1dc-48b1-8dfa-c755ed3c6512 HTTP/1.1" 200 - via_upstream - "-" 0 321 137 - "34.228.250.194,10.132.0.16" "python-requests/2.33.0" "cab44f62-ece8-4dbd-a627-7344c01c0ba8" "maas.apps.a3dc5cee932be05f4d88.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.37:54094 10.134.0.37:443 10.132.0.16:49720 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T08:55:04.490062Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-04T08:55:04.490089Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-04T08:55:04.490092Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-04T08:55:04.490096Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-11-04T08:55:04.422Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 77 68 - "34.228.250.194,10.132.0.16" "python-requests/2.33.0" "2796e75c-7930-42dc-9825-919da87666f4" "maas.apps.a3dc5cee932be05f4d88.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.37:443 10.132.0.16:47546 - v1-completions-path 2026-11-04T08:55:19.349939Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-04T08:55:19.197Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 152 - "34.228.250.194,10.132.0.16" "python-requests/2.33.0" "36b9c37b-3420-4c49-86d3-a8a08c91c887" "maas.apps.a3dc5cee932be05f4d88.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.37:54078 10.134.0.37:443 10.132.0.16:39856 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T08:55:19.984Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 164 - "34.228.250.194,10.132.0.16" "python-requests/2.33.0" "49bafc8d-8c89-4f99-98c2-967348f07615" "maas.apps.a3dc5cee932be05f4d88.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.37:34478 10.134.0.37:443 10.132.0.16:39864 - v1-completions-path [2026-11-04T08:55:20.772Z] "POST /maas-api/internal/v1/api-keys/validate HTTP/1.1" 404 NR route_not_found - "-" 31 0 11 - "34.228.250.194,10.132.0.16" "python-requests/2.33.0" "75e5144b-5353-4ab0-92f0-7e1a332f61e6" "maas.apps.a3dc5cee932be05f4d88.prod.konflux-ocp-ci.dev" "-" - - 10.134.0.37:443 10.132.0.16:39868 - - 2026-11-04T08:55:53.067007Z info ads ADS: new connection for node:3 2026-11-04T08:55:53.070124Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-04T08:55:53.070169Z info cache read certificate from file resource=file-root:system 2026-11-04T08:56:45.563440Z info ads ADS: "" 3 terminated with stream closed 2026-11-04T08:57:00.278516Z info ads ADS: new connection for node:4 2026-11-04T08:57:00.291681Z info cache read certificate from file resource=file-root:system 2026-11-04T08:57:02.095000Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:57:09.672562Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:57:13.727169Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T08:57:10.900Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 836 2826 - "34.228.250.194,10.132.0.16" "python-requests/2.33.0" "0ae98175-fba0-4cd6-ad58-462baa6f4985" "maas.apps.a3dc5cee932be05f4d88.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.37:38952 10.134.0.37:443 10.132.0.16:47496 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T08:57:17.073458Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:57:53.633036Z info ads ADS: "" 4 terminated 2026-11-04T08:57:53.703810Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:57:55.923955Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:58:08.939907Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8