2026-10-30T19:59:13.137970Z info FLAG: --concurrency="0" 2026-10-30T19:59:13.137990Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T19:59:13.137994Z info FLAG: --help="false" 2026-10-30T19:59:13.137997Z info FLAG: --log_as_json="false" 2026-10-30T19:59:13.137999Z info FLAG: --log_caller="" 2026-10-30T19:59:13.138001Z info FLAG: --log_output_level="default:info" 2026-10-30T19:59:13.138003Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T19:59:13.138007Z info FLAG: --log_target="[stdout]" 2026-10-30T19:59:13.138009Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T19:59:13.138011Z info FLAG: --outlierLogPath="" 2026-10-30T19:59:13.138013Z info FLAG: --profiling="true" 2026-10-30T19:59:13.138015Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T19:59:13.138017Z info FLAG: --proxyLogLevel="warning" 2026-10-30T19:59:13.138019Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T19:59:13.138021Z info FLAG: --stsPort="0" 2026-10-30T19:59:13.138023Z info FLAG: --templateFile="" 2026-10-30T19:59:13.138025Z info FLAG: --tokenManagerPlugin="" 2026-10-30T19:59:13.138027Z info FLAG: --vklog="0" 2026-10-30T19:59:13.138029Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T19:59:13.138349Z info Proxy role ips=[10.133.0.33] type=router id=maas-default-gateway-openshift-default-7f5667f86c-dh2bx.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T19:59:13.138410Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T19:59:13.139630Z info cpu limit detected as 2, setting concurrency 2026-10-30T19:59:13.140096Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T19:59:13.140118Z info JWT policy is third-party-jwt 2026-10-30T19:59:13.140123Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T19:59:13.141428Z info Starting default Istio SDS Server 2026-10-30T19:59:13.141451Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T19:59:13.141483Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T19:59:13.141850Z info Opening status port 15020 2026-10-30T19:59:13.142876Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T19:59:13.144032Z info sds Starting SDS grpc server 2026-10-30T19:59:13.144051Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T19:59:13.144715Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T19:59:13.145922Z info Starting proxy agent 2026-10-30T19:59:13.145964Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T19:59:13.219315Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T19:59:13.372407Z info ads ADS: new connection for node:1 2026-10-30T19:59:13.373151Z info ads ADS: new connection for node:2 2026-10-30T19:59:13.427718Z info cache generated new workload certificate resourceName=default latency=283.501877ms ttl=23h59m59.572286344s 2026-10-30T19:59:13.427760Z info cache Root cert has changed, start rotating root cert 2026-10-30T19:59:13.427802Z info cache returned workload trust anchor from cache ttl=23h59m59.572198231s 2026-10-30T19:59:13.427825Z info cache returned workload certificate from cache ttl=23h59m59.572175885s 2026-10-30T19:59:13.427954Z info cache returned workload trust anchor from cache ttl=23h59m59.572047391s 2026-10-30T19:59:13.428702Z info cache returned workload trust anchor from cache ttl=23h59m59.571298964s 2026-10-30T19:59:13.502753Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:59:20.636142Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:59:21.589992Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-10-30T19:59:22.293752Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:59:23.948273Z info Readiness succeeded in 10.813981419s 2026-10-30T19:59:23.948660Z info Envoy proxy is ready 2026-10-30T19:59:23.990140Z info Readiness succeeded in 10.85585251s 2026-10-30T19:59:24.039008Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T19:59:32.902052Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:00:07.750462Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:00:07.750503Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:00:07.750519Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:00:07.750524Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T20:00:07.663Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 568 86 - "34.228.250.194,10.132.0.13" "curl/7.76.1" "9ad86089-3dab-48a7-bfd0-3d73e5343522" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "10.133.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.33:38516 10.133.0.33:443 10.132.0.13:46856 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T20:00:07.802Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 441 69 - "34.228.250.194,10.132.0.13" "curl/7.76.1" "eda0d471-250c-41e9-855f-53ce40513689" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "10.133.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:46324 10.133.0.33:443 10.132.0.13:46866 - v1-model-routing [2026-10-30T20:00:08.393Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 53 - "34.228.250.194,10.132.0.13" "curl/7.76.1" "f073d7fe-470b-4143-8d8c-f4e517b239d3" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.33:443 10.132.0.13:46868 - v1-model-routing [2026-10-30T20:00:08.550Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 50 - "34.228.250.194,10.132.0.13" "curl/7.76.1" "ac3cc372-0e1e-4452-887d-21c33cef2660" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.33:443 10.132.0.13:46884 - v1-model-routing [2026-10-30T20:00:08.793Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 51 - "34.228.250.194,10.132.0.13" "curl/7.76.1" "ea453d9e-8c32-405e-ab41-f76d9409d9a5" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.33:443 10.132.0.13:46888 - v1-model-routing 2026-10-30T20:00:09.149447Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:00:09.149484Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:00:09.149499Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:00:09.149504Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T20:00:09.081Z] "GET /v1/subscriptions HTTP/1.1" 200 - via_upstream - "-" 0 488 67 - "34.228.250.194,10.132.0.13" "curl/7.76.1" "71c21264-3797-44f2-a368-34538ae067e8" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "10.133.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.33:38516 10.133.0.33:443 10.132.0.13:46894 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T20:02:40.263034Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:02:40.263080Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:02:40.263099Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T20:02:40.263106Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T20:02:40.168Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 95 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "91baed87-c3df-44ed-9457-63d31b149ff4" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "10.133.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.33:58666 10.133.0.33:443 10.132.0.13:48102 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T20:02:42.086334Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:02:42.086380Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:02:42.086402Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:02:42.086409Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T20:02:41.914Z] "DELETE /v1/api-keys/b3784058-683b-4193-bce2-0497659759fb HTTP/1.1" 200 - via_upstream - "-" 0 321 172 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "149a631c-ae82-4bb0-8c9d-c63b2b718a9c" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "10.133.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.33:58666 10.133.0.33:443 10.132.0.13:48124 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T20:02:42.471843Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:02:42.471879Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:02:42.471923Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:02:42.471930Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T20:02:42.262Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 209 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "7d52fa69-cc8a-96f6-8446-6165c12b79b3" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "10.133.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.33:58666 10.133.0.33:443 10.132.0.13:48136 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T20:02:43.172781Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:02:43.172819Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:02:43.172833Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:02:43.172838Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T20:02:42.862Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 310 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "6e5c766a-d404-4933-bddc-898097e57ce1" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "10.133.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.33:58666 10.133.0.33:443 10.132.0.13:48150 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T20:02:44.769697Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:02:44.769732Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:02:44.769746Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T20:02:44.769751Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T20:02:43.203Z] "DELETE /v1/api-keys/6e09f8db-035e-44d4-9363-175f4bb3ac88 HTTP/1.1" 200 - via_upstream - "-" 0 321 1566 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "8f3bb852-2f73-4f9f-a497-17c4ff59e971" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "10.133.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.33:58666 10.133.0.33:443 10.132.0.13:48160 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T20:02:45.263331Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:02:45.263434Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:02:45.263469Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-10-30T20:02:45.263511Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T20:02:45.263539Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:02:45.941084Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:02:45.941120Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:02:45.941136Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T20:02:45.941140Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:02:46.076116Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-10-30T20:02:46.097470Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:02:46.097518Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:02:46.097540Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:02:46.097547Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T20:02:44.988Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 274 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "9707cff5-963b-466b-abbb-0aae1b8a7848" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "10.133.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.33:58666 10.133.0.33:443 10.132.0.13:48170 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T20:02:45.740Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 200 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "4cfcf224-3237-422f-baec-353021c893e5" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "10.133.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.33:58666 10.133.0.33:443 10.132.0.13:48180 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T20:02:45.958Z] "DELETE /v1/api-keys/ce24806c-3dbe-436b-9121-ce97bdcb7510 HTTP/1.1" 200 - via_upstream - "-" 0 321 138 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "fa716a60-43d4-4e70-a42c-8b567b41762c" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "10.133.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.33:58682 10.133.0.33:443 10.132.0.13:48182 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T20:02:52.849473Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T20:02:52.290Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 288 65 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "eaaed990-0aca-4cba-ab56-beaaaf3d8742" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "10.133.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:46324 10.133.0.33:443 10.132.0.13:34218 - v1-completions-path [2026-10-30T20:02:53.110Z] "POST /maas-api/internal/v1/api-keys/validate HTTP/1.1" 404 NR route_not_found - "-" 31 0 19 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "e9c9ea77-aa45-4986-ad5d-eab5d47efb6f" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "-" - - 10.133.0.33:443 10.132.0.13:34236 - - 2026-10-30T20:03:09.216848Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:09:32.334627Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:09:58.821687Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:10:32.952756Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:10:32.952792Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:10:32.952806Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:10:32.952811Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T20:10:32.881Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 74 316 71 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "4c32cd0f-6ef7-410b-aa61-950d74e5b45f" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "10.133.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.33:34152 10.133.0.33:443 10.132.0.13:57174 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T20:10:40.706915Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:10:57.798288Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:11:01.918353Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:11:17.923265Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:11:33.574177Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T20:11:39.296Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 76 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "fa09d1c6-e185-4b3a-bc87-e22f646a04fc" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:42712 10.133.0.33:443 10.132.0.13:44082 - v1-completions-path 2026-10-30T20:11:57.325182Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:12:07.750541Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:12:11.997808Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:12:29.393081Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:12:31.629659Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:12:51.112097Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T20:12:52.933Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 278 65 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "6423887d-6648-4fe6-9ade-6d4a626ede0b" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "10.133.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:33136 10.133.0.33:443 10.132.0.13:54916 - v1-completions-path 2026-10-30T20:12:55.413744Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:13:13.176587Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:13:29.147601Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:13:39.997625Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:13:42.977458Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T20:13:59.060Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 36 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "00f020af-52d0-42b0-bc33-c35766472b11" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.33:443 10.132.0.13:55112 - v1-completions-path 2026-10-30T20:14:01.323405Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:14:04.002706Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:14:25.122179Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:14:27.859208Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:15:51.260464Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:15:51.260508Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:15:51.260528Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:15:51.260534Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T20:15:51.176Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 80 322 83 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "a1cb09a3-9edf-4aed-9157-96e9becf4918" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "10.133.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.33:33644 10.133.0.33:443 10.132.0.13:55754 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T20:16:01.928Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 43 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "1af5ba0d-6c92-4ab3-9e61-54b279278866" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.33:443 10.132.0.13:46262 - v1-completions-path 2026-10-30T20:16:04.120424Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:16:07.317475Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:17:38.681812Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:17:53.822974Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:17:55.936066Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:17:58.837565Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T20:17:58.837591Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T20:17:58.837594Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T20:17:58.837598Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-10-30T20:17:58.803Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 123 34 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "92f2277e-d19c-45e7-99fd-a27c2dda66f6" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.33:443 10.132.0.13:45224 - v1-completions-path 2026-10-30T20:18:10.671844Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:19:25.182520Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:19:25.182553Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:19:25.182568Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T20:19:25.182572Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T20:19:25.119Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 62 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "340d62ad-33e3-4ef2-855c-508b3035ce08" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "10.133.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.33:51520 10.133.0.33:443 10.132.0.13:50488 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T20:19:29.510401Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:19:46.374275Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:20:07.693414Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:20:21.851231Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:20:26.424553Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:20:38.391503Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:20:43.012373Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:21:00.925757Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:21:10.925306Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:21:16.446474Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:21:16.446511Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:21:16.446530Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:21:16.446535Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T20:21:16.554482Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-10-30T20:21:16.366Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 324 79 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "aa7b6d97-2d48-43a6-ac8a-2facc1de9f04" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "10.133.0.88:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.33:44696 10.133.0.33:443 10.132.0.13:52560 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T20:21:16.472Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 82 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "df0e4b15-9158-4067-8835-2cdfa9e455ce" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "10.132.0.47:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:60694 10.133.0.33:443 10.132.0.13:52576 - v1-catch-all-path 2026-10-30T20:21:19.367572Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:21:24.915544Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-10-30T20:21:24.856Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 59 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "9192f846-a486-4fd2-89e8-b2e9c2e6b213" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "10.132.0.47:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:60694 10.133.0.33:443 10.132.0.13:52378 - v1-catch-all-path 2026-10-30T20:21:25.092611Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-10-30T20:21:25.034Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 58 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "8c08615f-e111-474b-aed1-5b630385d8f2" "maas.apps.c1e65e8f3f6b6cbbf8ab.prod.konflux-ocp-ci.dev" "10.132.0.47:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:60694 10.133.0.33:443 10.132.0.13:52390 - v1-catch-all-path 2026-10-30T20:21:33.148469Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:21:35.565590Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:21:50.036315Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:22:01.022824Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8