2026-11-01T21:53:08.083859Z info FLAG: --concurrency="0" 2026-11-01T21:53:08.083879Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-01T21:53:08.083884Z info FLAG: --help="false" 2026-11-01T21:53:08.083887Z info FLAG: --log_as_json="false" 2026-11-01T21:53:08.083890Z info FLAG: --log_caller="" 2026-11-01T21:53:08.083893Z info FLAG: --log_output_level="default:info" 2026-11-01T21:53:08.083897Z info FLAG: --log_stacktrace_level="default:none" 2026-11-01T21:53:08.083903Z info FLAG: --log_target="[stdout]" 2026-11-01T21:53:08.083907Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-01T21:53:08.083910Z info FLAG: --outlierLogPath="" 2026-11-01T21:53:08.083912Z info FLAG: --profiling="true" 2026-11-01T21:53:08.083915Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-01T21:53:08.083918Z info FLAG: --proxyLogLevel="warning" 2026-11-01T21:53:08.083921Z info FLAG: --serviceCluster="istio-proxy" 2026-11-01T21:53:08.083924Z info FLAG: --stsPort="0" 2026-11-01T21:53:08.083927Z info FLAG: --templateFile="" 2026-11-01T21:53:08.083929Z info FLAG: --tokenManagerPlugin="" 2026-11-01T21:53:08.083933Z info FLAG: --vklog="0" 2026-11-01T21:53:08.083937Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-01T21:53:08.084133Z info Proxy role ips=[10.133.0.42] type=router id=maas-default-gateway-openshift-default-5bc884dd87-hcdq8.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-01T21:53:08.084194Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-01T21:53:08.085295Z info cpu limit detected as 2, setting concurrency 2026-11-01T21:53:08.085505Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-01T21:53:08.085519Z info JWT policy is third-party-jwt 2026-11-01T21:53:08.085524Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-01T21:53:08.086671Z info Starting default Istio SDS Server 2026-11-01T21:53:08.086697Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-01T21:53:08.086719Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-01T21:53:08.086970Z info Opening status port 15020 2026-11-01T21:53:08.088276Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-01T21:53:08.088615Z info sds Starting SDS grpc server 2026-11-01T21:53:08.088649Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-01T21:53:08.089435Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-01T21:53:08.090526Z info Starting proxy agent 2026-11-01T21:53:08.090557Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-01T21:53:08.155189Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-01T21:53:08.180531Z info cache generated new workload certificate resourceName=default latency=91.805208ms ttl=23h59m59.819471686s 2026-11-01T21:53:08.180570Z info cache Root cert has changed, start rotating root cert 2026-11-01T21:53:08.180608Z info cache returned workload trust anchor from cache ttl=23h59m59.819392671s 2026-11-01T21:53:08.291290Z info ads ADS: new connection for node:1 2026-11-01T21:53:08.291345Z info cache returned workload certificate from cache ttl=23h59m59.708656081s 2026-11-01T21:53:08.291450Z info ads ADS: new connection for node:2 2026-11-01T21:53:08.291524Z info cache returned workload trust anchor from cache ttl=23h59m59.708477911s 2026-11-01T21:53:08.374261Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T21:53:16.251932Z info Readiness succeeded in 8.170824416s 2026-11-01T21:53:16.252384Z info Envoy proxy is ready 2026-11-01T21:53:26.411764Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T21:53:26.411804Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T21:53:26.411817Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T21:53:26.411821Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-01T21:53:26.321Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 320 90 - "44.212.242.249,10.132.0.10" "curl/7.76.1" "4b5d4218-3b04-447e-bd19-11cd6812efbc" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.42:47792 10.133.0.42:443 10.132.0.10:41270 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T21:53:26.720Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 442 93 - "44.212.242.249,10.132.0.10" "curl/7.76.1" "37b457af-fd35-4787-8046-23786a9c1756" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.133.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.42:44394 10.133.0.42:443 10.132.0.10:41272 - v1-model-routing [2026-11-01T21:53:26.907Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 354 56 - "44.212.242.249,10.132.0.10" "curl/7.76.1" "59d6b418-4330-4cdf-86af-b91c244abd76" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.133.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.42:44404 10.133.0.42:443 10.132.0.10:41288 - v1-model-routing [2026-11-01T21:53:28.166Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 52 - "44.212.242.249,10.132.0.10" "curl/7.76.1" "7b971009-03fb-4aa6-a93f-e534ec8165da" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.42:443 10.132.0.10:41296 - v1-model-routing 2026-11-01T21:53:28.637764Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T21:53:28.637799Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T21:53:28.637810Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-01T21:53:28.637815Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-01T21:53:28.628Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 91 9 - "44.212.242.249,10.132.0.10" "curl/7.76.1" "b106ab62-2ddf-4796-89b3-a61c87c483ba" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.42:47806 10.133.0.42:443 10.132.0.10:41312 - odh-ai-gateway-infra.maas-api-route.4 2026-11-01T21:55:49.809546Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T21:55:49.809603Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T21:55:49.809626Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T21:55:49.809633Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T21:55:50.303813Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T21:55:50.303927Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T21:55:50.303976Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T21:55:50.304006Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-01T21:55:49.738Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 71 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "64e848ac-9f39-434d-b4c3-3d2a9d8a36f3" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.42:45400 10.133.0.42:443 10.132.0.10:34138 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T21:55:50.215Z] "DELETE /v1/api-keys/85084e5b-36b1-42a2-9025-f997d55617a4 HTTP/1.1" 200 - via_upstream - "-" 0 321 88 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "1de1d2da-89fd-4243-93e0-9bf33ab9a68d" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.42:45412 10.133.0.42:443 10.132.0.10:34144 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T21:55:50.592789Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T21:55:50.592900Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T21:55:50.592951Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T21:55:50.592982Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-01T21:55:50.528Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 64 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "20afda30-12d7-461a-a869-fb0e7ebf431a" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.42:45412 10.133.0.42:443 10.132.0.10:34156 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T21:55:58.470Z] "POST /maas-api/internal/v1/api-keys/validate HTTP/1.1" 404 NR route_not_found - "-" 31 0 7 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "7b78d897-a48d-402f-87d4-81bda7538e85" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "-" - - 10.133.0.42:443 10.132.0.10:40192 - - 2026-11-01T21:56:07.086709Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T21:56:21.292296Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T21:58:43.058618Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T21:58:43.058659Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T21:58:43.058675Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T21:58:43.058680Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-01T21:58:42.991Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 67 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "d55fcbe4-b8c6-437c-aad8-431dfb3afdc8" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.42:45454 10.133.0.42:443 10.132.0.10:42156 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T21:58:43.302Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 136 334 79 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "c05efb2d-c591-4041-b2bd-72677fbc4e2c" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.133.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.42:44394 10.133.0.42:443 10.132.0.10:42166 - v1-chat-completions-path 2026-11-01T21:59:00.163910Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T21:59:00.163943Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T21:59:00.163957Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T21:59:00.163961Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-01T21:59:00.082Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 322 81 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "58085a73-c371-42fb-b898-4056225df57e" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.42:49986 10.133.0.42:443 10.132.0.10:45064 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T22:00:07.363434Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:00:46.059340Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:01:49.817018Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:03:42.030560Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:04:05.337857Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:04:44.407421Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:06:27.448781Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:07:26.221815Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:09:29.147223Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:24:28.978875Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 [2026-11-01T22:26:19.167Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 110 321 149 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "5d99f4e4-ab01-4f1b-bcfe-cf0617b38646" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.133.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.42:44394 10.133.0.42:443 10.132.0.10:44596 - v1-chat-completions-path 2026-11-01T22:26:22.527098Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:26:22.527130Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:26:22.527141Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:26:22.527146Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T22:26:22.527284Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-01T22:26:22.527312Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 2026-11-01T22:26:22.527359Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-01T22:26:22.527367Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-01T22:26:22.527370Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-01T22:26:22.527373Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-11-01T22:26:22.327Z] "POST /v1/chat/completions HTTP/1.1" 404 - via_upstream - "-" 120 126 200 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "a05fa9bc-6d61-495b-818e-956c4dbd7eb0" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.133.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.42:44394 10.133.0.42:443 10.132.0.10:44624 - v1-chat-completions-path 2026-11-01T22:33:10.018167Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:33:31.632193Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:33:43.113136Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:33:43.113167Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:33:43.113179Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:33:43.113184Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-01T22:33:43.039Z] "POST /v1/api-keys HTTP/1.1" 400 - via_upstream - "-" 80 94 73 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "a7270358-4651-48a5-95d3-705d3844e57e" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.42:41972 10.133.0.42:443 10.132.0.10:51314 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T22:34:18.591883Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:34:22.917622Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:34:22.917658Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:34:22.917671Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:34:22.917676Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-01T22:34:22.858Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 87 330 58 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "6cdae6d7-7cd7-4ad8-881b-19e23a0b87f3" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.42:41972 10.133.0.42:443 10.132.0.10:33804 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T22:34:22.969Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 77 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "d6914b9c-8f44-411c-828f-20f16c7bfb3b" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.134.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.42:53498 10.133.0.42:443 10.132.0.10:33812 - v1-completions-path [2026-11-01T22:34:23.447Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 58 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "149677aa-8876-45a1-836e-c91eb36b95a3" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.134.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.42:53508 10.133.0.42:443 10.132.0.10:33826 - v1-completions-path 2026-11-01T22:34:35.665849Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:34:37.482718Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:34:54.491615Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:34:56.386483Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:34:59.310714Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:35:07.164292Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T22:35:14.348Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 60 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "ff1740f2-7144-46bd-b445-03160be1e933" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.134.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.42:53508 10.133.0.42:443 10.132.0.10:49462 - v1-completions-path [2026-11-01T22:35:14.915Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 56 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "48599717-0435-48b1-b276-e01971f03ec5" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.134.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.42:53498 10.133.0.42:443 10.132.0.10:49470 - v1-completions-path 2026-11-01T22:35:31.368295Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:35:41.586964Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:35:43.214848Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T22:35:50.256Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 60 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "8bab5459-4546-4d0d-a158-f03551d0f9ab" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.134.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.42:53508 10.133.0.42:443 10.132.0.10:50308 - v1-completions-path 2026-11-01T22:36:02.144274Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:36:03.851182Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:36:20.138255Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:36:20.229387Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-01T22:36:20.067Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 287 166 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "49ac43d6-d4c6-4186-9538-360de82f6e32" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.133.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.42:44404 10.133.0.42:443 10.132.0.10:48954 - v1-completions-path 2026-11-01T22:36:22.272270Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:36:38.854171Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:36:40.572599Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:36:53.711205Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=3 [2026-11-01T22:36:55.660Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 289 63 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "00cc1120-5395-4095-9ec7-ecbee244c9f7" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.133.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.42:44394 10.133.0.42:443 10.132.0.10:54128 - v1-completions-path 2026-11-01T22:36:56.618767Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:37:07.236511Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:37:09.934214Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:37:25.725018Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:37:27.838977Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:37:35.333346Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:37:53.656143Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:38:03.743125Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:39:03.182006Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:39:05.537185Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:40:45.375111Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:40:48.256858Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:42:13.873865Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:42:28.763569Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:42:30.520375Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:42:47.678809Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:44:14.961917Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:44:29.536401Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:44:44.396357Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:44:44.396404Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:44:44.396426Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T22:44:44.396431Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-01T22:44:44.341Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 628 55 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "148b73ef-d133-48be-a435-ad9192c974dd" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.42:34798 10.133.0.42:443 10.132.0.10:37078 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T22:44:45.740861Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:44:57.337215Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:45:00.831670Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:45:09.737457Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:45:12.077467Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:45:18.366382Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:45:18.366419Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:45:18.366434Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:45:18.366439Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-01T22:45:18.314Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 524 51 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "c509ecfe-626f-4674-9875-e74fc4cb78aa" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.42:34798 10.133.0.42:443 10.132.0.10:37252 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T22:45:29.831293Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:48:16.771552Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:48:19.625214Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-11-01T22:48:19.522Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 102 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "e5989a47-692d-400b-a44d-b88931f05691" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.42:47496 10.133.0.42:443 10.132.0.10:33846 - v1-catch-all-path 2026-11-01T22:48:21.877248Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T22:48:25.405Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 429 - - "-" 61 18 56 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "76727c0d-e7b9-4303-9759-ac91555d27a8" "maas.apps.e566464fd7d0f4c757f5.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.42:443 10.132.0.10:45414 - v1-catch-all-path 2026-11-01T22:48:36.931158Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:48:38.510173Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:48:52.837550Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:49:03.638136Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8