2026-11-04T08:53:25.332739Z info FLAG: --concurrency="0" 2026-11-04T08:53:25.332757Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-04T08:53:25.332761Z info FLAG: --help="false" 2026-11-04T08:53:25.332763Z info FLAG: --log_as_json="false" 2026-11-04T08:53:25.332765Z info FLAG: --log_caller="" 2026-11-04T08:53:25.332767Z info FLAG: --log_output_level="default:info" 2026-11-04T08:53:25.332770Z info FLAG: --log_stacktrace_level="default:none" 2026-11-04T08:53:25.332774Z info FLAG: --log_target="[stdout]" 2026-11-04T08:53:25.332777Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-04T08:53:25.332779Z info FLAG: --outlierLogPath="" 2026-11-04T08:53:25.332781Z info FLAG: --profiling="true" 2026-11-04T08:53:25.332783Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-04T08:53:25.332785Z info FLAG: --proxyLogLevel="warning" 2026-11-04T08:53:25.332787Z info FLAG: --serviceCluster="istio-proxy" 2026-11-04T08:53:25.332789Z info FLAG: --stsPort="0" 2026-11-04T08:53:25.332791Z info FLAG: --templateFile="" 2026-11-04T08:53:25.332793Z info FLAG: --tokenManagerPlugin="" 2026-11-04T08:53:25.332807Z info FLAG: --vklog="0" 2026-11-04T08:53:25.332809Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-04T08:53:25.332974Z info Proxy role ips=[10.133.0.34] type=router id=maas-default-gateway-openshift-default-855f9574cf-hd5fh.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-04T08:53:25.333035Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-04T08:53:25.334540Z info cpu limit detected as 2, setting concurrency 2026-11-04T08:53:25.334772Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-04T08:53:25.334784Z info JWT policy is third-party-jwt 2026-11-04T08:53:25.334786Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-04T08:53:25.336773Z info Starting default Istio SDS Server 2026-11-04T08:53:25.336808Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-04T08:53:25.336833Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-04T08:53:25.337267Z info Opening status port 15020 2026-11-04T08:53:25.338197Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-04T08:53:25.339013Z info sds Starting SDS grpc server 2026-11-04T08:53:25.339082Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-04T08:53:25.339804Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-04T08:53:25.340707Z info Starting proxy agent 2026-11-04T08:53:25.340739Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-04T08:53:25.394312Z info cache generated new workload certificate resourceName=default latency=55.154849ms ttl=23h59m59.605693147s 2026-11-04T08:53:25.394357Z info cache Root cert has changed, start rotating root cert 2026-11-04T08:53:25.394402Z info cache returned workload trust anchor from cache ttl=23h59m59.605598827s 2026-11-04T08:53:25.409249Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-04T08:53:25.537014Z info ads ADS: new connection for node:1 2026-11-04T08:53:25.537082Z info cache returned workload certificate from cache ttl=23h59m59.462920686s 2026-11-04T08:53:25.537483Z info ads ADS: new connection for node:2 2026-11-04T08:53:25.537562Z info cache returned workload trust anchor from cache ttl=23h59m59.462439596s 2026-11-04T08:53:25.586073Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:53:31.910769Z info Readiness succeeded in 6.58132238s 2026-11-04T08:53:31.911093Z info Envoy proxy is ready 2026-11-04T08:53:53.360362Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:53:58.126283Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:54:01.758724Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:54:06.292898Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:54:09.319954Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:54:12.867210Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:54:16.773938Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:54:41.060726Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T08:54:40.929Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 131 - "34.228.250.194,10.132.0.16" "curl/7.76.1" "e9eaff8d-9783-4b5d-a7a1-4560f0e33aa1" "maas.apps.a3dc5cee932be05f4d88.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.34:47500 10.133.0.34:443 10.132.0.16:34374 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T08:54:41.265Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 412 133 - "34.228.250.194,10.132.0.16" "curl/7.76.1" "a10b86c9-55e9-4f8c-8d85-f738fdadfa7e" "maas.apps.a3dc5cee932be05f4d88.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.34:53668 10.133.0.34:443 10.132.0.16:34388 - v1-model-routing [2026-11-04T08:54:42.038Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 100 - "34.228.250.194,10.132.0.16" "curl/7.76.1" "b0e74073-4233-4051-8e9c-7313c5b8c05f" "maas.apps.a3dc5cee932be05f4d88.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.34:443 10.132.0.16:34408 - v1-model-routing [2026-11-04T08:54:42.166Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 102 - "34.228.250.194,10.132.0.16" "curl/7.76.1" "c7b88b77-3afe-4cb7-9126-fbe8d142bb29" "maas.apps.a3dc5cee932be05f4d88.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.34:443 10.132.0.16:34422 - v1-model-routing [2026-11-04T08:55:00.820Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 113 - "34.228.250.194,10.132.0.16" "python-requests/2.33.0" "5ff2a1c6-dcd6-4232-a1a2-a3da01b25585" "maas.apps.a3dc5cee932be05f4d88.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.133.0.34:443 10.132.0.16:35774 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T08:55:02.163633Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-04T08:55:01.756Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 86 - "34.228.250.194,10.132.0.16" "python-requests/2.33.0" "b17c9cc6-fb27-46e1-aa81-3d73f1e3b1c2" "maas.apps.a3dc5cee932be05f4d88.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.133.0.34:443 10.132.0.16:35792 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T08:55:02.030Z] "DELETE /v1/api-keys/a7f383eb-6be2-4057-8234-80566291a388 HTTP/1.1" 200 - via_upstream - "-" 0 321 133 - "34.228.250.194,10.132.0.16" "python-requests/2.33.0" "3c5263ad-8ab4-4c9b-a349-41fd805ffc51" "maas.apps.a3dc5cee932be05f4d88.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.34:35962 10.133.0.34:443 10.132.0.16:35794 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T08:55:02.950714Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-04T08:55:02.821Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 129 - "34.228.250.194,10.132.0.16" "python-requests/2.33.0" "a3830e7a-e954-43ac-9ae7-6e94ad418105" "maas.apps.a3dc5cee932be05f4d88.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.34:35962 10.133.0.34:443 10.132.0.16:35810 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T08:55:12.834Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 299 130 - "34.228.250.194,10.132.0.16" "python-requests/2.33.0" "57c1b8b5-0e56-4fc0-8a81-bf2ef2294d90" "maas.apps.a3dc5cee932be05f4d88.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.34:53668 10.133.0.34:443 10.132.0.16:36150 - v1-completions-path 2026-11-04T08:55:20.839181Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-04T08:55:20.827Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 91 11 - "34.228.250.194,10.132.0.16" "python-requests/2.33.0" "3259346c-0e8d-4c35-9fb8-505ed12301c3" "maas.apps.a3dc5cee932be05f4d88.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.34:35962 10.133.0.34:443 10.132.0.16:56550 - odh-ai-gateway-infra.maas-api-route.4 2026-11-04T08:55:53.079214Z info ads ADS: new connection for node:3 2026-11-04T08:55:53.084718Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-04T08:55:53.084832Z info cache read certificate from file resource=file-root:system 2026-11-04T08:56:45.560431Z info ads ADS: "" 3 terminated 2026-11-04T08:57:00.334439Z info ads ADS: new connection for node:4 2026-11-04T08:57:00.341649Z info cache read certificate from file resource=file-root:system 2026-11-04T08:57:02.143467Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:57:09.678714Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:57:13.774179Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-04T08:57:13.760Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 91 13 - "34.228.250.194,10.132.0.16" "python-requests/2.33.0" "a408dc69-53ae-47aa-8299-62c2e46a9150" "maas.apps.a3dc5cee932be05f4d88.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.34:60522 10.133.0.34:443 10.132.0.16:60832 - odh-ai-gateway-infra.maas-api-route.4 2026-11-04T08:57:17.091638Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:57:24.451684Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: gRPC status code is not OK thread=25 [2026-11-04T08:57:22.438Z] "POST /maas-api/v1/api-keys HTTP/1.1" 500 - - "-" 66 23 2013 - "34.228.250.194,10.132.0.16" "python-requests/2.33.0" "1a2fe953-e100-447b-a2c1-d33b0c3c98f8" "maas.apps.a3dc5cee932be05f4d88.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.133.0.34:443 10.132.0.16:60840 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T08:57:53.643106Z info ads ADS: "" 4 terminated 2026-11-04T08:57:53.723779Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:57:55.949059Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T08:58:08.946203Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8