2026-11-01T22:34:31.739697Z info FLAG: --concurrency="0" 2026-11-01T22:34:31.739717Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-01T22:34:31.739721Z info FLAG: --help="false" 2026-11-01T22:34:31.739723Z info FLAG: --log_as_json="false" 2026-11-01T22:34:31.739725Z info FLAG: --log_caller="" 2026-11-01T22:34:31.739727Z info FLAG: --log_output_level="default:info" 2026-11-01T22:34:31.739729Z info FLAG: --log_stacktrace_level="default:none" 2026-11-01T22:34:31.739734Z info FLAG: --log_target="[stdout]" 2026-11-01T22:34:31.739736Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-01T22:34:31.739738Z info FLAG: --outlierLogPath="" 2026-11-01T22:34:31.739740Z info FLAG: --profiling="true" 2026-11-01T22:34:31.739742Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-01T22:34:31.739744Z info FLAG: --proxyLogLevel="warning" 2026-11-01T22:34:31.739746Z info FLAG: --serviceCluster="istio-proxy" 2026-11-01T22:34:31.739749Z info FLAG: --stsPort="0" 2026-11-01T22:34:31.739750Z info FLAG: --templateFile="" 2026-11-01T22:34:31.739752Z info FLAG: --tokenManagerPlugin="" 2026-11-01T22:34:31.739755Z info FLAG: --vklog="0" 2026-11-01T22:34:31.739757Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-01T22:34:31.739979Z info Proxy role ips=[10.132.0.50] type=router id=maas-default-gateway-openshift-default-6886b96f9-bftkj.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-01T22:34:31.740029Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-01T22:34:31.741037Z info cpu limit detected as 2, setting concurrency 2026-11-01T22:34:31.741229Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-01T22:34:31.741240Z info JWT policy is third-party-jwt 2026-11-01T22:34:31.741242Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-01T22:34:31.742715Z info Starting default Istio SDS Server 2026-11-01T22:34:31.742743Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-01T22:34:31.742761Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-01T22:34:31.743112Z info Opening status port 15020 2026-11-01T22:34:31.743992Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-01T22:34:31.744624Z info sds Starting SDS grpc server 2026-11-01T22:34:31.744661Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-01T22:34:31.745428Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-01T22:34:31.746208Z info Starting proxy agent 2026-11-01T22:34:31.746238Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-01T22:34:31.809387Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-01T22:34:31.811587Z info cache generated new workload certificate resourceName=default latency=66.823595ms ttl=23h59m59.188416636s 2026-11-01T22:34:31.811623Z info cache Root cert has changed, start rotating root cert 2026-11-01T22:34:31.811653Z info cache returned workload trust anchor from cache ttl=23h59m59.188348918s 2026-11-01T22:34:31.906050Z info ads ADS: new connection for node:1 2026-11-01T22:34:31.906112Z info cache returned workload certificate from cache ttl=23h59m59.093890911s 2026-11-01T22:34:31.906150Z info ads ADS: new connection for node:2 2026-11-01T22:34:31.906218Z info cache returned workload trust anchor from cache ttl=23h59m59.093783292s 2026-11-01T22:34:31.986090Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:34:39.793937Z info Readiness succeeded in 8.057297561s 2026-11-01T22:34:39.794350Z info Envoy proxy is ready 2026-11-01T22:36:51.099803Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:36:51.099865Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:36:51.099888Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T22:36:51.099894Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-01T22:36:50.998Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 101 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "fbc849b7-a58f-4459-b9f0-5a4ad6984edc" "maas.apps.c70f7758cc01335924b7.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.50:47158 10.132.0.50:443 10.132.0.17:46540 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T22:36:52.810937Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T22:36:52.810976Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T22:36:52.810995Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T22:36:52.811002Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-01T22:36:52.735Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 316 75 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "74d7621b-83ad-4d54-b60c-978870ea3f7b" "maas.apps.c70f7758cc01335924b7.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.50:47170 10.132.0.50:443 10.132.0.17:46554 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T22:36:57.862Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 46 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "7ec82f5b-bc9c-4cbe-be06-9fd85e04c158" "maas.apps.c70f7758cc01335924b7.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.50:443 10.132.0.17:38620 - v1-completions-path 2026-11-01T22:36:59.381121Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T22:36:59.381155Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T22:36:59.381168Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T22:36:59.381172Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-01T22:36:59.322Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 58 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "b2576245-867b-4e02-9a9d-b60d7e12687d" "maas.apps.c70f7758cc01335924b7.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.50:47170 10.132.0.50:443 10.132.0.17:38632 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T22:37:00.905471Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T22:37:00.905517Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T22:37:00.905538Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-01T22:37:00.905544Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-01T22:37:00.849Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 56 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "fdaf8c6c-a25e-41fe-9166-7181c7f1f639" "maas.apps.c70f7758cc01335924b7.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.50:47170 10.132.0.50:443 10.132.0.17:38640 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T22:37:11.430575Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:37:26.812187Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:41:16.944445Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:43:08.135029Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:43:35.462873Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:44:16.261840Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:44:53.596722Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:45:55.501745Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:48:06.868842Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:49:36.468314Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:05:37.678949Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 [2026-11-01T23:08:11.820Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 110 327 270 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "380bbe91-2f10-4d2c-aa28-3187dabd5c35" "maas.apps.c70f7758cc01335924b7.prod.konflux-ocp-ci.dev" "10.133.0.39:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.50:40404 10.132.0.50:443 10.132.0.17:56332 - v1-chat-completions-path 2026-11-01T23:08:15.184374Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T23:08:15.184417Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T23:08:15.184434Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T23:08:15.184440Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T23:08:15.184618Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 2026-11-01T23:08:15.184677Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-01T23:08:15.184688Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-01T23:08:15.184692Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-01T23:08:15.184697Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-11-01T23:08:15.120Z] "POST /v1/chat/completions HTTP/1.1" 404 - via_upstream - "-" 120 126 63 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "fabaa7a5-3376-415c-9c51-3e69beab792d" "maas.apps.c70f7758cc01335924b7.prod.konflux-ocp-ci.dev" "10.133.0.39:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.50:58304 10.132.0.50:443 10.132.0.17:37448 - v1-chat-completions-path 2026-11-01T23:13:03.359581Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:13:23.329367Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:13:34.301713Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T23:13:34.301745Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T23:13:34.301756Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T23:13:34.301761Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-01T23:13:34.240Z] "POST /v1/api-keys HTTP/1.1" 400 - via_upstream - "-" 80 94 61 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "e686bb6b-dab4-4574-8e5e-81a0938b465a" "maas.apps.c70f7758cc01335924b7.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.50:47572 10.132.0.50:443 10.132.0.17:40090 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T23:14:06.691725Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-01T23:14:06.691748Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-01T23:14:06.691751Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-01T23:14:06.691755Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-11-01T23:14:06.647Z] "POST /llm/premium-simulated-simulated-premium/v1/completions HTTP/1.1" 403 - - "-" 74 77 43 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "36ed1152-f289-42c2-80e6-cb251a95b499" "maas.apps.c70f7758cc01335924b7.prod.konflux-ocp-ci.dev" "-" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.50:443 10.132.0.17:52670 - v1-completions-path 2026-11-01T23:14:09.424276Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:14:24.767128Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:14:28.201681Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:14:45.035651Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=3 2026-11-01T23:14:45.559180Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:14:47.301094Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:14:49.997053Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:14:58.445360Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:15:04.083516Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T23:15:04.083556Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T23:15:04.083569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T23:15:04.083574Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-01T23:15:04.017Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 77 319 65 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "82100fae-d8c4-4bd8-a0d7-7a5351055913" "maas.apps.c70f7758cc01335924b7.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.50:35630 10.132.0.50:443 10.132.0.17:34934 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T23:15:05.296Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 57 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "beb9a3a3-3ffe-4767-a072-4fd4f50e7ed0" "maas.apps.c70f7758cc01335924b7.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.50:443 10.132.0.17:50326 - v1-completions-path 2026-11-01T23:15:07.200387Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-11-01T23:15:07.133Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 67 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "39c48a0d-0c89-499a-b3ef-2bbbf5bfd96d" "maas.apps.c70f7758cc01335924b7.prod.konflux-ocp-ci.dev" "10.134.0.30:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.50:44572 10.132.0.50:443 10.132.0.17:50344 - v1-completions-path 2026-11-01T23:15:22.739050Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:15:32.978255Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:15:36.657962Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T23:15:39.779Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 59 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "3e43b33e-376c-441c-a356-555a0fef1af6" "maas.apps.c70f7758cc01335924b7.prod.konflux-ocp-ci.dev" "10.134.0.30:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.50:44572 10.132.0.50:443 10.132.0.17:33050 - v1-completions-path 2026-11-01T23:15:52.818461Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:15:54.379413Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:16:10.714645Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:16:12.699018Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:16:29.284425Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:16:30.879884Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:16:46.974686Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:16:57.612105Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:17:00.217704Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:17:03.930662Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T23:17:03.930704Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T23:17:03.930717Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T23:17:03.930721Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-01T23:17:03.866Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 322 64 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "54636340-995d-4a36-b74b-efe3b2e8d9f4" "maas.apps.c70f7758cc01335924b7.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.50:40598 10.132.0.50:443 10.132.0.17:42198 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T23:17:16.127326Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:17:18.160294Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T23:17:21.743Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 33 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "dbc62035-e6d3-446c-9602-d1f76c99ce44" "maas.apps.c70f7758cc01335924b7.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.50:443 10.132.0.17:49810 - v1-completions-path 2026-11-01T23:17:25.704147Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:17:43.809377Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:17:54.087183Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T23:18:08.111Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 36 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "3f6cab61-3580-41e0-9002-df4bc3d6a7f5" "maas.apps.c70f7758cc01335924b7.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.50:443 10.132.0.17:41850 - v1-completions-path 2026-11-01T23:18:09.636942Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:18:12.294743Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:19:47.717842Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:19:50.438804Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:21:13.960573Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:21:16.684688Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-01T23:21:16.353Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 73 302 331 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "837e85aa-bbdb-4576-818f-dc3cf8ef1605" "maas.apps.c70f7758cc01335924b7.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||e2e-trlp-test-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.50:40586 10.132.0.50:443 10.132.0.17:35680 - v1-completions-path 2026-11-01T23:21:28.944285Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:21:30.675239Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:21:46.638696Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:23:07.864349Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T23:23:07.864402Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T23:23:07.864429Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T23:23:07.864436Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-01T23:23:07.796Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 68 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "69ddd748-16ad-4101-a18c-be9e3a91b73c" "maas.apps.c70f7758cc01335924b7.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.50:39590 10.132.0.50:443 10.132.0.17:32848 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T23:23:13.134513Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:23:29.806094Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:23:45.269471Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:23:57.043761Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:24:00.443006Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:24:11.735787Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:24:14.072649Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:24:19.847201Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T23:24:19.847243Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T23:24:19.847260Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T23:24:19.847265Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-01T23:24:19.791Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 524 56 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "1d05b861-d376-4277-a604-17a8820fead1" "maas.apps.c70f7758cc01335924b7.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.50:46640 10.132.0.50:443 10.132.0.17:42234 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T23:24:20.074Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 60 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "97ad65f2-8e4a-46d5-8794-07d8dab0f939" "maas.apps.c70f7758cc01335924b7.prod.konflux-ocp-ci.dev" "10.134.0.30:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.50:37754 10.132.0.50:443 10.132.0.17:42238 - v1-completions-path [2026-11-01T23:24:20.307Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 52 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "8bc62a5a-5810-4aa6-bb06-e749a3f2ff58" "maas.apps.c70f7758cc01335924b7.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.50:443 10.132.0.17:42248 - v1-completions-path 2026-11-01T23:24:31.937558Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:27:17.308456Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:27:22.372793Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:27:25.825338Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-11-01T23:27:25.753Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 71 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "1c5f20ae-51b2-4337-b6ff-58dedccc3e19" "maas.apps.c70f7758cc01335924b7.prod.konflux-ocp-ci.dev" "10.133.0.41:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.50:34310 10.132.0.50:443 10.132.0.17:55994 - v1-catch-all-path 2026-11-01T23:27:37.595922Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:27:39.130067Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:27:53.568566Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:28:04.372437Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8