2026-11-01T22:30:23.185284Z info FLAG: --concurrency="0" 2026-11-01T22:30:23.185301Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-01T22:30:23.185305Z info FLAG: --help="false" 2026-11-01T22:30:23.185307Z info FLAG: --log_as_json="false" 2026-11-01T22:30:23.185309Z info FLAG: --log_caller="" 2026-11-01T22:30:23.185311Z info FLAG: --log_output_level="default:info" 2026-11-01T22:30:23.185313Z info FLAG: --log_stacktrace_level="default:none" 2026-11-01T22:30:23.185318Z info FLAG: --log_target="[stdout]" 2026-11-01T22:30:23.185320Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-01T22:30:23.185322Z info FLAG: --outlierLogPath="" 2026-11-01T22:30:23.185324Z info FLAG: --profiling="true" 2026-11-01T22:30:23.185326Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-01T22:30:23.185328Z info FLAG: --proxyLogLevel="warning" 2026-11-01T22:30:23.185330Z info FLAG: --serviceCluster="istio-proxy" 2026-11-01T22:30:23.185332Z info FLAG: --stsPort="0" 2026-11-01T22:30:23.185334Z info FLAG: --templateFile="" 2026-11-01T22:30:23.185336Z info FLAG: --tokenManagerPlugin="" 2026-11-01T22:30:23.185339Z info FLAG: --vklog="0" 2026-11-01T22:30:23.185341Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-01T22:30:23.185482Z info Proxy role ips=[10.133.0.43] type=router id=maas-default-gateway-openshift-default-58847f9fc5-8m4jm.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-01T22:30:23.185536Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-01T22:30:23.186605Z info cpu limit detected as 2, setting concurrency 2026-11-01T22:30:23.186866Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-01T22:30:23.186880Z info JWT policy is third-party-jwt 2026-11-01T22:30:23.186884Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-01T22:30:23.188538Z info Starting default Istio SDS Server 2026-11-01T22:30:23.188566Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-01T22:30:23.188585Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-01T22:30:23.188875Z info Opening status port 15020 2026-11-01T22:30:23.190060Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-01T22:30:23.191284Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-01T22:30:23.192033Z info Starting proxy agent 2026-11-01T22:30:23.192062Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-01T22:30:23.201346Z info sds Starting SDS grpc server 2026-11-01T22:30:23.201369Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-01T22:30:23.252556Z info cache generated new workload certificate resourceName=default latency=62.086641ms ttl=23h59m59.747446384s 2026-11-01T22:30:23.252591Z info cache Root cert has changed, start rotating root cert 2026-11-01T22:30:23.252623Z info cache returned workload trust anchor from cache ttl=23h59m59.747377886s 2026-11-01T22:30:23.255019Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-01T22:30:23.368615Z info ads ADS: new connection for node:1 2026-11-01T22:30:23.368839Z info cache returned workload certificate from cache ttl=23h59m59.631162287s 2026-11-01T22:30:23.369315Z info ads ADS: new connection for node:2 2026-11-01T22:30:23.369607Z info cache returned workload trust anchor from cache ttl=23h59m59.630394386s 2026-11-01T22:30:23.449588Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:30:31.322749Z info Readiness succeeded in 8.140321219s 2026-11-01T22:30:31.323068Z info Envoy proxy is ready [2026-11-01T22:30:54.999Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 65 - "44.212.242.249,10.132.0.11" "curl/7.76.1" "6f6f17a0-d0d3-459c-875a-5406d0b76b34" "maas.apps.3e43e8909c74f3327723.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.43:443 10.132.0.11:46028 - v1-model-routing 2026-11-01T22:30:55.790190Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=26 2026-11-01T22:30:55.790214Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=26 2026-11-01T22:30:55.790217Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=26 2026-11-01T22:30:55.790222Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=26 [2026-11-01T22:30:55.770Z] "POST /v1/chat/completions HTTP/1.1" 401 - - "-" 122 88 19 - "44.212.242.249,10.132.0.11" "curl/7.76.1" "1ef4fdc8-7d63-4e02-b17f-2bca06c7913d" "maas.apps.3e43e8909c74f3327723.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.43:443 10.132.0.11:46044 - v1-model-routing 2026-11-01T22:33:13.284077Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-01T22:33:13.284129Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-01T22:33:13.284145Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-01T22:33:13.284150Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-01T22:33:13.202Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 81 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "cfb8e8af-af09-46a0-be2e-5d18d1bd32f6" "maas.apps.3e43e8909c74f3327723.prod.konflux-ocp-ci.dev" "10.132.0.45:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.43:47720 10.133.0.43:443 10.132.0.11:51476 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T22:33:13.906572Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-01T22:33:13.906603Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-01T22:33:13.906616Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-01T22:33:13.906621Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-01T22:33:14.122862Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:33:14.122903Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:33:14.122922Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:33:14.122928Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T22:33:14.242298Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:33:14.242328Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:33:14.242341Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:33:14.242346Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-01T22:33:13.843Z] "DELETE /v1/api-keys/a69e1e55-b800-4484-90d6-9793316c5a71 HTTP/1.1" 200 - via_upstream - "-" 0 321 63 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "969b9557-80da-4af3-b5fd-f2545e5a67c1" "maas.apps.3e43e8909c74f3327723.prod.konflux-ocp-ci.dev" "10.132.0.45:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.43:47720 10.133.0.43:443 10.132.0.11:51484 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T22:33:14.058Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 64 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "84f8615b-de01-4141-8758-038410192272" "maas.apps.3e43e8909c74f3327723.prod.konflux-ocp-ci.dev" "10.132.0.45:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.43:47732 10.133.0.43:443 10.132.0.11:51496 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T22:33:14.179Z] "DELETE /v1/api-keys/3445c5cc-261a-4798-80c0-03f5c892e9aa HTTP/1.1" 200 - via_upstream - "-" 0 321 62 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "cc024551-2260-4e3e-89e0-43c3321b46b2" "maas.apps.3e43e8909c74f3327723.prod.konflux-ocp-ci.dev" "10.132.0.45:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.43:47732 10.133.0.43:443 10.132.0.11:51500 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T22:33:14.389622Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-01T22:33:14.389656Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-01T22:33:14.389669Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-01T22:33:14.389674Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-01T22:33:14.605831Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:33:14.605880Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:33:14.605901Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T22:33:14.605908Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-01T22:33:14.324Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 65 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "9dc04278-7c00-4f28-8e78-1e70dc93079f" "maas.apps.3e43e8909c74f3327723.prod.konflux-ocp-ci.dev" "10.132.0.45:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.43:47720 10.133.0.43:443 10.132.0.11:51502 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T22:33:14.541Z] "DELETE /v1/api-keys/dcfda481-24d9-4f06-a4e0-c7f2d6d918e1 HTTP/1.1" 200 - via_upstream - "-" 0 321 64 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "d5ed4f6f-bd0f-4286-9d42-f6cf1cf4983d" "maas.apps.3e43e8909c74f3327723.prod.konflux-ocp-ci.dev" "10.132.0.45:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.43:47732 10.133.0.43:443 10.132.0.11:51504 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T22:33:16.615Z] "POST /maas-api/internal/v1/api-keys/validate HTTP/1.1" 404 NR route_not_found - "-" 31 0 7 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "b4fabe6b-1f06-466d-bacd-7daa6e7941f0" "maas.apps.3e43e8909c74f3327723.prod.konflux-ocp-ci.dev" "-" - - 10.133.0.43:443 10.132.0.11:51520 - - [2026-11-01T22:33:20.865Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 288 75 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "bba60452-74af-40ef-a8ee-d95027bd52e4" "maas.apps.3e43e8909c74f3327723.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.43:58928 10.133.0.43:443 10.132.0.11:40690 - v1-completions-path 2026-11-01T22:33:31.000067Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:33:38.234249Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T22:35:54.282Z] "GET /v1/tenants HTTP/1.1" 404 NR route_not_found - "-" 0 0 14 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "e5b1fd33-8d31-4e4d-aa25-fd088264a71c" "maas.apps.3e43e8909c74f3327723.prod.konflux-ocp-ci.dev" "-" - - 10.133.0.43:443 10.132.0.11:54212 - - 2026-11-01T22:36:39.291266Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:39:14.356174Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:39:48.534009Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:41:07.725108Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:41:51.246848Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:44:27.828321Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:45:48.923690Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:46:22.173337Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:51:03.178144Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:51:27.214228Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:52:09.987801Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-01T22:52:09.987830Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-01T22:52:09.987834Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-01T22:52:09.987839Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-11-01T22:52:09.950Z] "POST /llm/premium-simulated-simulated-premium/v1/completions HTTP/1.1" 403 - - "-" 74 77 37 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "23372683-95a2-4e4f-aa05-f5334c1dca55" "maas.apps.3e43e8909c74f3327723.prod.konflux-ocp-ci.dev" "-" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.43:443 10.132.0.11:37050 - v1-completions-path 2026-11-01T22:52:12.449613Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:52:14.990942Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-01T22:52:14.990981Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-01T22:52:14.990995Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-01T22:52:14.991000Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 [2026-11-01T22:52:14.901Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 87 329 89 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "f1e5a664-7b4d-47c0-9658-8b86f947255b" "maas.apps.3e43e8909c74f3327723.prod.konflux-ocp-ci.dev" "10.132.0.45:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.43:35944 10.133.0.43:443 10.132.0.11:37066 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T22:52:15.444Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 72 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "22c4b966-3b7a-410e-8141-b1b399466e10" "maas.apps.3e43e8909c74f3327723.prod.konflux-ocp-ci.dev" "10.134.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.43:40720 10.133.0.43:443 10.132.0.11:37072 - v1-completions-path 2026-11-01T22:52:27.896969Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:52:29.658722Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:52:46.565744Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:52:48.197099Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:52:50.813806Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:52:59.362446Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:53:06.946545Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-11-01T22:53:06.328Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 58 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "f3e1c4fc-d185-4e13-af93-e02f944a6795" "maas.apps.3e43e8909c74f3327723.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.43:443 10.132.0.11:52572 - v1-completions-path [2026-11-01T22:53:06.556Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 59 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "9fea1f7c-35f2-4761-8e58-9cc9777ebe83" "maas.apps.3e43e8909c74f3327723.prod.konflux-ocp-ci.dev" "10.134.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.43:40720 10.133.0.43:443 10.132.0.11:52584 - v1-completions-path [2026-11-01T22:53:06.894Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 56 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "c84d7e8d-217d-4a9e-ad5c-c68ff6ea8659" "maas.apps.3e43e8909c74f3327723.prod.konflux-ocp-ci.dev" "10.134.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.43:40720 10.133.0.43:443 10.132.0.11:52596 - v1-completions-path [2026-11-01T22:53:08.116Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 60 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "c2a671c6-8249-4a1d-824a-16c32c8f31ed" "maas.apps.3e43e8909c74f3327723.prod.konflux-ocp-ci.dev" "10.134.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.43:34278 10.133.0.43:443 10.132.0.11:52614 - v1-completions-path 2026-11-01T22:53:23.553351Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:53:33.826884Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:53:35.459744Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T22:53:41.165Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 65 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "c4a4b4b4-a309-4606-b6bd-ca1b2ea389ac" "maas.apps.3e43e8909c74f3327723.prod.konflux-ocp-ci.dev" "10.134.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.43:40720 10.133.0.43:443 10.132.0.11:46632 - v1-completions-path 2026-11-01T22:53:54.383126Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:53:55.957716Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:54:00.846488Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:54:00.846524Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:54:00.846540Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T22:54:00.846545Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-01T22:54:00.776Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 73 315 69 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "def13509-b8f1-47fc-a96b-b27507d7e256" "maas.apps.3e43e8909c74f3327723.prod.konflux-ocp-ci.dev" "10.132.0.45:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.43:54960 10.133.0.43:443 10.132.0.11:55216 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T22:54:12.452385Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:54:14.483953Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:54:31.167601Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:54:32.778099Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:54:48.943454Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:54:59.548143Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:55:02.098404Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:55:18.027090Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:55:20.137609Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T22:55:23.725Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 34 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "5e37917b-16bb-99db-9ae7-19fd04a9d58c" "maas.apps.3e43e8909c74f3327723.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.43:443 10.132.0.11:38208 - v1-completions-path 2026-11-01T22:55:27.662972Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:55:45.842280Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:55:56.153635Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T22:58:19.185532Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-01T23:01:13.311894Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:01:15.572785Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:02:54.054455Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:02:56.755206Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:04:30.424150Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:04:45.684751Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:04:47.487610Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:05:06.447351Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:06:31.713280Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:06:49.144777Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:07:04.571898Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:07:16.466859Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:07:19.889891Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:07:30.833820Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:07:33.217233Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:07:51.130544Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:10:41.135637Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:10:46.270500Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:10:49.573583Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=26 [2026-11-01T23:10:49.496Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 77 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "481c05af-8dc4-4c38-ae56-27fea6a967f4" "maas.apps.3e43e8909c74f3327723.prod.konflux-ocp-ci.dev" "10.134.0.35:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.43:58758 10.133.0.43:443 10.132.0.11:37406 - v1-catch-all-path [2026-11-01T23:10:50.009Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 429 - - "-" 61 18 58 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "fdafa0cb-fb6e-43b1-ae46-1db7b61ea7e9" "maas.apps.3e43e8909c74f3327723.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.43:443 10.132.0.11:37408 - v1-catch-all-path 2026-11-01T23:11:01.628470Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:11:03.287875Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:11:17.866169Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:11:28.608856Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8