2026-11-03T06:03:11.619369Z info FLAG: --concurrency="0" 2026-11-03T06:03:11.619397Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T06:03:11.619404Z info FLAG: --help="false" 2026-11-03T06:03:11.619407Z info FLAG: --log_as_json="false" 2026-11-03T06:03:11.619410Z info FLAG: --log_caller="" 2026-11-03T06:03:11.619413Z info FLAG: --log_output_level="default:info" 2026-11-03T06:03:11.619416Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T06:03:11.619423Z info FLAG: --log_target="[stdout]" 2026-11-03T06:03:11.619426Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T06:03:11.619429Z info FLAG: --outlierLogPath="" 2026-11-03T06:03:11.619433Z info FLAG: --profiling="true" 2026-11-03T06:03:11.619435Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T06:03:11.619438Z info FLAG: --proxyLogLevel="warning" 2026-11-03T06:03:11.619441Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T06:03:11.619444Z info FLAG: --stsPort="0" 2026-11-03T06:03:11.619447Z info FLAG: --templateFile="" 2026-11-03T06:03:11.619451Z info FLAG: --tokenManagerPlugin="" 2026-11-03T06:03:11.619455Z info FLAG: --vklog="0" 2026-11-03T06:03:11.619460Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T06:03:11.619714Z info Proxy role ips=[10.132.0.49] type=router id=maas-default-gateway-openshift-default-778b7b5c47-8mbmr.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T06:03:11.619857Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T06:03:11.621604Z info cpu limit detected as 2, setting concurrency 2026-11-03T06:03:11.621941Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T06:03:11.621959Z info JWT policy is third-party-jwt 2026-11-03T06:03:11.621963Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T06:03:11.624896Z info Starting default Istio SDS Server 2026-11-03T06:03:11.624961Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T06:03:11.625003Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T06:03:11.625192Z info Opening status port 15020 2026-11-03T06:03:11.625857Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T06:03:11.627089Z info sds Starting SDS grpc server 2026-11-03T06:03:11.627174Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T06:03:11.627490Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T06:03:11.628522Z info Starting proxy agent 2026-11-03T06:03:11.628598Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T06:03:11.695188Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T06:03:11.717427Z info cache generated new workload certificate resourceName=default latency=90.161933ms ttl=23h59m59.282577236s 2026-11-03T06:03:11.717460Z info cache Root cert has changed, start rotating root cert 2026-11-03T06:03:11.717492Z info cache returned workload trust anchor from cache ttl=23h59m59.28250839s 2026-11-03T06:03:11.814769Z info ads ADS: new connection for node:1 2026-11-03T06:03:11.814871Z info cache returned workload certificate from cache ttl=23h59m59.185132609s 2026-11-03T06:03:11.815885Z info ads ADS: new connection for node:2 2026-11-03T06:03:11.815968Z info cache returned workload trust anchor from cache ttl=23h59m59.184033321s 2026-11-03T06:03:11.900735Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:03:20.492438Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:03:21.401543Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T06:03:22.211445Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T06:03:22.702399Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:03:22.733041Z info Readiness succeeded in 11.117887868s 2026-11-03T06:03:22.733478Z info Envoy proxy is ready [2026-11-03T06:03:52.391Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 358 78 - "44.212.242.249,10.133.0.12" "curl/7.76.1" "52c5c4ce-d48d-4b60-b023-16be76a79c96" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "10.132.0.47:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.49:40210 10.132.0.49:443 10.133.0.12:58092 - v1-model-routing 2026-11-03T06:03:53.552485Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T06:03:53.552523Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T06:03:53.552536Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T06:03:53.552541Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T06:03:52.901Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 53 - "44.212.242.249,10.133.0.12" "curl/7.76.1" "d4a4e86d-b6c5-45f7-882c-31b85bfc4f13" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.49:443 10.133.0.12:58094 - v1-model-routing [2026-11-03T06:03:53.482Z] "GET /v1/subscriptions HTTP/1.1" 200 - via_upstream - "-" 0 488 70 - "44.212.242.249,10.133.0.12" "curl/7.76.1" "316eaf69-84c1-41e6-b13c-d5aa59305722" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.49:48116 10.132.0.49:443 10.133.0.12:58098 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T06:06:43.183752Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T06:06:43.183786Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T06:06:43.183812Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T06:06:43.183818Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T06:06:43.102Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 84 327 81 - "44.212.242.249,10.133.0.12" "python-requests/2.34.2" "6c0f20e5-b689-4647-934b-af3226ce7d1f" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.49:44380 10.132.0.49:443 10.133.0.12:46750 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T06:06:44.080030Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T06:06:44.080066Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T06:06:44.080082Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T06:06:44.080088Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T06:06:44.495946Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T06:06:44.495975Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T06:06:44.495986Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T06:06:44.495990Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T06:06:44.567091Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-03T06:06:44.567176Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T06:06:44.567201Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T06:06:44.567221Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T06:06:44.567231Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T06:06:44.686021Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T06:06:44.686054Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T06:06:44.686065Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T06:06:44.686070Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T06:06:44.000Z] "DELETE /v1/api-keys/cc23df42-2274-40c3-b8d8-4b7c6a6bea91 HTTP/1.1" 200 - via_upstream - "-" 0 321 79 - "44.212.242.249,10.133.0.12" "python-requests/2.34.2" "a2a02f76-a460-4550-b9ef-eab20a53c3a7" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.49:44380 10.132.0.49:443 10.133.0.12:46756 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T06:06:44.407Z] "DELETE /v1/api-keys/f4a1168b-c002-4482-aaff-5045284da8fa HTTP/1.1" 200 - via_upstream - "-" 0 321 88 - "44.212.242.249,10.133.0.12" "python-requests/2.34.2" "c90035c6-9bb0-400e-8888-ccea21300d52" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.49:44380 10.132.0.49:443 10.133.0.12:46760 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T06:06:44.484Z] "DELETE /v1/api-keys/e19a1e31-d698-4e7f-b897-1f8e17215adf HTTP/1.1" 200 - via_upstream - "-" 0 321 82 - "44.212.242.249,10.133.0.12" "python-requests/2.34.2" "4a10d033-e1bc-429c-ba3d-095fd235907c" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.49:44380 10.132.0.49:443 10.133.0.12:46762 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T06:06:44.604Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 81 - "44.212.242.249,10.133.0.12" "python-requests/2.34.2" "6681367f-932d-439e-a689-ccfc09403c9e" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.49:44384 10.132.0.49:443 10.133.0.12:46764 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T06:06:44.877972Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T06:06:44.878004Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T06:06:44.878016Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T06:06:44.878021Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T06:06:45.051097Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T06:06:45.051131Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T06:06:45.051143Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T06:06:45.051148Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T06:06:45.119002Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T06:06:45.119026Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T06:06:45.119029Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T06:06:45.119032Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 2026-11-03T06:06:45.373128Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-11-03T06:06:45.376504Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T06:06:45.376585Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T06:06:45.376609Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T06:06:45.376617Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T06:06:44.800Z] "DELETE /v1/api-keys/9aa5ac6a-4ea0-4e38-a61e-5448560b33cc HTTP/1.1" 200 - via_upstream - "-" 0 321 77 - "44.212.242.249,10.133.0.12" "python-requests/2.34.2" "aa851269-4dbd-47c0-82d6-ea7e0cec6f90" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.49:44384 10.132.0.49:443 10.133.0.12:46780 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T06:06:44.969Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 81 - "44.212.242.249,10.133.0.12" "python-requests/2.34.2" "a07a14a9-1c9b-4b48-8639-dd931e6ce3c3" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.49:44384 10.132.0.49:443 10.133.0.12:46788 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T06:06:45.082Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 77 36 - "44.212.242.249,10.133.0.12" "python-requests/2.34.2" "609ee925-b16e-491f-8323-ac5b748067da" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.49:443 10.133.0.12:46802 - v1-completions-path [2026-11-03T06:06:45.301Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 74 - "44.212.242.249,10.133.0.12" "python-requests/2.34.2" "d2ca28dd-0f46-4879-89bc-f4559deedcc6" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.49:44384 10.132.0.49:443 10.133.0.12:46818 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T06:06:47.829Z] "POST /maas-api/internal/v1/api-keys/cleanup HTTP/1.1" 404 NR route_not_found - "-" 2 0 9 - "44.212.242.249,10.133.0.12" "python-requests/2.34.2" "8158dbf5-6782-4efa-9c4d-df4176914b7a" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "-" - - 10.132.0.49:443 10.133.0.12:46822 - - [2026-11-03T06:06:50.710Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 295 61 - "44.212.242.249,10.133.0.12" "python-requests/2.34.2" "a4346036-fdc7-41c6-9a41-fdd7f85afc3b" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "10.132.0.47:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.49:43546 10.132.0.49:443 10.133.0.12:53030 - v1-completions-path 2026-11-03T06:07:03.021386Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:07:17.778344Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:11:50.605068Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:12:10.536684Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:13:01.750344Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:13:19.011031Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:13:23.660059Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:13:42.223012Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:13:44.615635Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:13:48.051176Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:13:58.061740Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T06:14:03.898Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 59 - "44.212.242.249,10.133.0.12" "python-requests/2.34.2" "2a73d393-1b8e-458b-a7fb-b3ec147a6f67" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.49:443 10.133.0.12:49604 - v1-completions-path [2026-11-03T06:14:04.222Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 67 - "44.212.242.249,10.133.0.12" "python-requests/2.34.2" "25b74102-0981-412a-ad60-0fc6f8f96220" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "10.133.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.49:54874 10.132.0.49:443 10.133.0.12:49616 - v1-completions-path [2026-11-03T06:14:05.226Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 56 - "44.212.242.249,10.133.0.12" "python-requests/2.34.2" "f072011a-e6af-4a80-8fb8-7c323f9e4759" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "10.133.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.49:54874 10.132.0.49:443 10.133.0.12:49628 - v1-completions-path 2026-11-03T06:14:21.106323Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:14:31.506826Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:14:36.046730Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:14:56.227918Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:14:58.688164Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T06:15:05.666Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 74 289 79 - "44.212.242.249,10.133.0.12" "python-requests/2.34.2" "9b4e07ba-9565-49f0-ba4a-0373eb053451" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "10.133.0.35:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.132.0.49:40444 10.132.0.49:443 10.133.0.12:40798 - v1-completions-path 2026-11-03T06:15:17.785176Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:15:22.371636Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:15:40.897753Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:15:43.236321Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:16:01.145203Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:16:11.849341Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:16:15.142245Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T06:16:34.019Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 37 - "44.212.242.249,10.133.0.12" "python-requests/2.34.2" "16f9ad3f-540c-4b3a-8550-c44d0a9e238f" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.49:443 10.133.0.12:47446 - v1-completions-path 2026-11-03T06:16:36.371639Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:16:39.361355Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:16:46.441229Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T06:16:46.441264Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T06:16:46.441277Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T06:16:46.441282Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T06:16:46.363Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 89 332 77 - "44.212.242.249,10.133.0.12" "python-requests/2.34.2" "c4712884-ac5f-4d79-980b-bb21bd0e3e19" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.49:34726 10.132.0.49:443 10.133.0.12:44168 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T06:16:46.461Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 62 - "44.212.242.249,10.133.0.12" "python-requests/2.34.2" "35f88609-d9f2-4402-b07b-1ca116188725" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "10.132.0.47:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.49:40210 10.132.0.49:443 10.133.0.12:44176 - v1-completions-path 2026-11-03T06:16:53.560758Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T06:16:59.248Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 299 63 - "44.212.242.249,10.133.0.12" "python-requests/2.34.2" "be52c349-98ab-4afc-b857-e98f5ca14c27" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "10.133.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.49:52222 10.132.0.49:443 10.133.0.12:33698 - v1-completions-path 2026-11-03T06:17:11.650787Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:17:22.025937Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:17:41.054987Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:17:44.816939Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:19:29.146229Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:19:32.920125Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:21:03.085346Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:21:17.875736Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:21:20.194224Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:21:39.154983Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:23:11.189711Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:23:27.816357Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:23:49.106015Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:24:03.349539Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:24:07.563848Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:24:18.438397Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:24:22.121323Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:24:41.989928Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:27:32.830353Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:27:38.323029Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T06:27:38.323070Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T06:27:38.323086Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T06:27:38.323093Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T06:27:38.253Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 324 69 - "44.212.242.249,10.133.0.12" "python-requests/2.34.2" "660713b1-7000-466a-9baa-f88ffe199a98" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "10.134.0.92:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.49:49964 10.132.0.49:443 10.133.0.12:44040 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T06:27:41.255150Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:27:46.988437Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-11-03T06:27:46.910Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 77 - "44.212.242.249,10.133.0.12" "python-requests/2.34.2" "498553af-fdf9-4577-a80c-b8d21c46cca3" "maas.apps.a3c09cf78a357df5f249.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.49:59506 10.132.0.49:443 10.133.0.12:46886 - v1-catch-all-path 2026-11-03T06:27:59.044663Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:28:01.181856Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:28:16.299782Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T06:28:27.205874Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9