2026-10-30T01:47:55.959452Z info FLAG: --concurrency="0" 2026-10-30T01:47:55.959469Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T01:47:55.959473Z info FLAG: --help="false" 2026-10-30T01:47:55.959476Z info FLAG: --log_as_json="false" 2026-10-30T01:47:55.959478Z info FLAG: --log_caller="" 2026-10-30T01:47:55.959480Z info FLAG: --log_output_level="default:info" 2026-10-30T01:47:55.959482Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T01:47:55.959486Z info FLAG: --log_target="[stdout]" 2026-10-30T01:47:55.959489Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T01:47:55.959491Z info FLAG: --outlierLogPath="" 2026-10-30T01:47:55.959493Z info FLAG: --profiling="true" 2026-10-30T01:47:55.959495Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T01:47:55.959497Z info FLAG: --proxyLogLevel="warning" 2026-10-30T01:47:55.959499Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T01:47:55.959501Z info FLAG: --stsPort="0" 2026-10-30T01:47:55.959503Z info FLAG: --templateFile="" 2026-10-30T01:47:55.959505Z info FLAG: --tokenManagerPlugin="" 2026-10-30T01:47:55.959507Z info FLAG: --vklog="0" 2026-10-30T01:47:55.959510Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T01:47:55.959646Z info Proxy role ips=[10.134.0.47] type=router id=maas-default-gateway-openshift-default-75d7c785f-nzg5g.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T01:47:55.959690Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T01:47:55.961060Z info cpu limit detected as 2, setting concurrency 2026-10-30T01:47:55.961268Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T01:47:55.961279Z info JWT policy is third-party-jwt 2026-10-30T01:47:55.961282Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T01:47:55.962517Z info Starting default Istio SDS Server 2026-10-30T01:47:55.962537Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T01:47:55.962554Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T01:47:55.962875Z info Opening status port 15020 2026-10-30T01:47:55.964196Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T01:47:55.964621Z info sds Starting SDS grpc server 2026-10-30T01:47:55.964708Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T01:47:55.965511Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T01:47:55.966265Z info Starting proxy agent 2026-10-30T01:47:55.966295Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T01:47:56.028656Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T01:47:56.041225Z info cache generated new workload certificate resourceName=default latency=76.41954ms ttl=23h59m59.958778665s 2026-10-30T01:47:56.041260Z info cache Root cert has changed, start rotating root cert 2026-10-30T01:47:56.041295Z info cache returned workload trust anchor from cache ttl=23h59m59.958706236s 2026-10-30T01:47:56.136897Z info ads ADS: new connection for node:1 2026-10-30T01:47:56.136977Z info cache returned workload certificate from cache ttl=23h59m59.863025039s 2026-10-30T01:47:56.137532Z info ads ADS: new connection for node:2 2026-10-30T01:47:56.137608Z info cache returned workload trust anchor from cache ttl=23h59m59.862392975s 2026-10-30T01:47:56.218811Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:48:04.048378Z info Readiness succeeded in 8.091534589s 2026-10-30T01:48:04.048713Z info Envoy proxy is ready 2026-10-30T01:54:02.079031Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:54:19.059748Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:55:49.701456Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:56:28.906706Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T01:56:28.906744Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T01:56:28.906759Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T01:56:28.906764Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T01:56:28.821Z] "DELETE /v1/api-keys/0fa804ea-b14b-4267-bd07-86ac0d8505d8 HTTP/1.1" 200 - via_upstream - "-" 0 321 85 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "38506d60-8a3c-47d9-93b7-e7f1dbeb20e6" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.47:42784 10.134.0.47:443 10.132.0.13:37294 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T01:56:29.199952Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T01:56:29.200075Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T01:56:29.200108Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T01:56:29.200116Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T01:56:29.115Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 84 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "c6ef2439-df6a-480a-adf3-4999210403a2" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.47:42784 10.134.0.47:443 10.132.0.13:37306 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T01:56:30.184672Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T01:56:30.184711Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T01:56:30.184726Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T01:56:30.184731Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-10-30T01:56:30.108Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 76 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "31c76a2c-8f60-424d-9d4b-6b186d2a8f7e" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.47:42798 10.134.0.47:443 10.132.0.13:32976 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T01:56:31.010985Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T01:56:31.011023Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T01:56:31.011039Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T01:56:31.011044Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T01:56:30.834Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 176 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "b26465eb-1fff-4d1a-8ff0-a93ea6f81aaf" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.47:42784 10.134.0.47:443 10.132.0.13:32980 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T01:56:31.656976Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T01:56:31.657014Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T01:56:31.657033Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T01:56:31.657040Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-10-30T01:56:31.583Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 73 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "838b9e41-9937-4d97-a060-54e9f8027757" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.47:42798 10.134.0.47:443 10.132.0.13:32996 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T01:56:32.728998Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:57:08.048192Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T01:57:08.048227Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T01:57:08.048244Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-10-30T01:57:08.048249Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 [2026-10-30T01:57:07.975Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 72 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "6c488dff-f403-4fd8-bbe0-fbfc352d57de" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.47:42798 10.134.0.47:443 10.132.0.13:47522 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T02:05:01.779Z] "POST /maas-api/v1/tokens HTTP/1.1" 401 - - "-" 20 88 24 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "abf06998-6c24-41c2-8e01-00fb258fc570" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.134.0.47:443 10.132.0.13:34878 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T02:05:05.519829Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T02:05:05.519872Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T02:05:05.519886Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-10-30T02:05:05.519891Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 [2026-10-30T02:05:05.450Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 69 312 69 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "aa872c3e-98a3-47ad-8cb6-c2bcd5e930e8" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.47:33956 10.134.0.47:443 10.132.0.13:34890 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T02:05:05.634Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 110 329 74 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "f591ff05-0ad4-4f82-8421-3f08dd55148b" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.44:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.47:38758 10.134.0.47:443 10.132.0.13:34902 - v1-chat-completions-path [2026-10-30T02:12:53.014Z] "GET /v1/tenants HTTP/1.1" 404 NR route_not_found - "-" 0 0 8 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "abab6954-3243-4145-ada1-466a4813d1df" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "-" - - 10.134.0.47:443 10.132.0.13:40156 - - 2026-10-30T02:15:33.358401Z info ads ADS: new connection for node:3 2026-10-30T02:15:33.368919Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-10-30T02:15:33.369044Z info cache read certificate from file resource=file-root:system 2026-10-30T02:15:38.804248Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-10-30T02:20:13.189071Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:20:26.768164Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:21:01.319300Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:21:04.355514Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T02:21:04.355631Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T02:21:04.355680Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T02:21:04.355711Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-10-30T02:21:04.023Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 66 308 332 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "622e30bd-ec87-45cf-8220-99c8c0f6939c" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.47:46630 10.134.0.47:443 10.132.0.13:49472 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T02:21:16.948218Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:21:20.796405Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:21:25.573612Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:21:32.435953Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:21:38.679584Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:21:42.230795Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:21:44.502513Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:21:51.688208Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:22:52.921656Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:23:16.091680Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:23:33.016861Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:23:35.918862Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:23:53.729469Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:25:08.670441Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T02:25:11.883Z] "POST /llm/e2e-dft-b75196/anything/e2e-dft-b75196-50d82c HTTP/1.1" 401 - - "-" 109 88 65 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "afea9fee-4594-4e12-8740-7273c09a7b17" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-b75196.llm.svc.cluster.local - 10.134.0.47:443 10.132.0.13:44370 - llm.external-model-e2e-dft-b75196-e05659e96551.2 2026-10-30T02:25:18.763071Z info ads ADS: new connection for node:4 2026-10-30T02:25:18.763689Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T02:25:18.763851Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T02:25:24.449758Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:25:52.403728Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:27:37.473014Z info ads ADS: "" 4 terminated 2026-10-30T02:28:01.076270Z info ads ADS: "" 3 terminated with stream closed 2026-10-30T02:35:12.986616Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:36:44.869102Z info cache event for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt : REMOVE, pushing to proxy 2026-10-30T02:36:47.812400Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:38:26.207850Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:40:16.107402Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:47:45.382893Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=3 2026-10-30T02:48:42.112154Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:49:05.513070Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:49:38.828629Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:49:41.730954Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=26 [2026-10-30T02:49:41.675Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 2817 3089 79 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "40dfb468-b45e-4d4e-bbbc-f862e3e5bfbf" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.134.0.47:45522 10.134.0.47:443 10.132.0.13:48130 - v1-chat-completions-path 2026-10-30T02:49:54.501263Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:49:57.614416Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:50:03.787658Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:50:10.941330Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:50:20.542483Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:50:35.590314Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:50:46.629831Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:50:50.239665Z info ads ADS: new connection for node:5 2026-10-30T02:50:50.239847Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T02:50:50.239875Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T02:50:52.710610Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:51:30.813951Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:51:49.886085Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:51:49.914554Z info ads ADS: "" 5 terminated 2026-10-30T02:51:52.243323Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-10-30T02:51:52.182Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 289 60 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "761f9bf9-108c-4d33-aef0-6a45d170788d" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.44:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.47:38758 10.134.0.47:443 10.132.0.13:39226 - v1-completions-path 2026-10-30T02:51:53.218009Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:52:01.176352Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:53:30.052321Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:53:47.027968Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:54:05.818875Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T02:54:05.818913Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T02:54:05.818932Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-10-30T02:54:05.818936Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 [2026-10-30T02:54:05.729Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 80 322 89 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "b0fbdb8b-4eb2-4e4c-a6ec-120ddadfcd85" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.47:52796 10.134.0.47:443 10.132.0.13:52880 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T02:54:07.978186Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:54:19.937510Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:54:26.804222Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:54:38.730375Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:54:42.246708Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T02:54:45.280Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 74 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "b72ca244-c156-44a0-86b7-563926cc77d8" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.43:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.47:48314 10.134.0.47:443 10.132.0.13:53572 - v1-completions-path 2026-10-30T02:54:57.614938Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T02:54:58.851Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 64 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "af8fd461-14e6-423d-b03f-c84a6ea5d64a" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.44:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.47:38758 10.134.0.47:443 10.132.0.13:44196 - v1-completions-path [2026-10-30T02:55:11.822Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 289 67 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "f3661c62-d85c-4aea-9eef-a9968dd644fe" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.44:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.47:38758 10.134.0.47:443 10.132.0.13:49068 - v1-completions-path 2026-10-30T02:55:42.138778Z info ads ADS: new connection for node:6 2026-10-30T02:55:42.144254Z info cache read certificate from file resource=file-root:system 2026-10-30T02:55:43.162399Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:55:44.750735Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:55:49.378488Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"1067","StreamId":"5068304297484912658"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=26 [2026-10-30T02:55:49.314Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 64 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "57e1666c-fdbc-4f7a-99f2-7d08f713d162" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.134.0.47:443 10.132.0.13:36584 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-10-30T02:55:55.701660Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T02:55:55.701696Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T02:55:55.701707Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-10-30T02:55:55.701712Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T02:55:55.701877Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-10-30T02:55:55.594Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1718 107 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "8dfcf3d1-bb11-49b4-8a66-f3259b2e1f6f" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.47:35442 10.134.0.47:443 10.132.0.13:53324 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T02:56:08.600019Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T02:56:08.600051Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T02:56:08.600062Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T02:56:08.600066Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-10-30T02:56:08.600240Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-10-30T02:56:08.498Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1713 101 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "18e6b13c-8e74-4d75-a1a2-0e95613f19ae" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.47:35442 10.134.0.47:443 10.132.0.13:52898 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T02:56:12.838172Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T02:56:12.838204Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T02:56:12.838214Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-10-30T02:56:12.838218Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T02:56:12.838378Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-10-30T02:56:12.743Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1712 94 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "a592f7c4-040e-4504-a6c8-a88d2aa71fe9" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.47:35442 10.134.0.47:443 10.132.0.13:50498 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T02:56:55.392622Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T02:56:55.392660Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T02:56:55.392670Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-10-30T02:56:55.392674Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T02:56:55.392861Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-10-30T02:56:55.288Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1720 104 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "83823137-84d9-44eb-ad12-3e77722956d3" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.47:35442 10.134.0.47:443 10.132.0.13:36456 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T02:57:08.165476Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T02:57:08.165515Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T02:57:08.165532Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T02:57:08.165537Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T02:57:08.165746Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T02:57:08.052Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1712 112 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "b93fa116-1017-465b-a79b-99a1f1295634" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.47:42562 10.134.0.47:443 10.132.0.13:57840 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T02:57:18.837420Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T02:57:18.837455Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T02:57:18.837466Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T02:57:18.837471Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-10-30T02:57:18.837662Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-10-30T02:57:18.735Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1722 102 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "bea773b0-bf49-440c-a436-702289cddc79" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.47:35442 10.134.0.47:443 10.132.0.13:35060 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T02:57:34.390607Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:57:34.485971Z info ads ADS: "" 6 terminated 2026-10-30T02:57:35.915969Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:57:42.832833Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:57:58.659360Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:58:03.190790Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:58:18.475326Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:58:34.092316Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T02:58:41.522Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 84 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "0774aba1-13e0-4613-81a6-37d971283023" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.43:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.47:41902 10.134.0.47:443 10.132.0.13:49832 - v1-completions-path 2026-10-30T02:58:57.502087Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:59:07.898631Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:59:12.532870Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T02:59:17.255Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 62 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "b54a3e5f-d0f3-461f-9be2-e263d4478105" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.43:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.47:48314 10.134.0.47:443 10.132.0.13:51816 - v1-completions-path 2026-10-30T02:59:29.540523Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:59:31.882260Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:59:38.922461Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:59:51.187658Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T02:59:56.473776Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:00:04.762342Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:00:10.009269Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:00:29.242312Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T03:00:32.334Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 66 18 58 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "ce61861e-bc5d-4f6d-9c2d-2c94c155b718" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.47:443 10.132.0.13:42934 - v1-completions-path 2026-10-30T03:00:32.710848Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:00:48.417414Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:00:50.174857Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T03:00:50.174895Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T03:00:50.174911Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T03:00:50.174917Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-10-30T03:00:50.093Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 89 332 81 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "21ae2781-7853-4305-beab-203a35c11151" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.47:53844 10.134.0.47:443 10.132.0.13:37536 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T03:00:51.306660Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T03:00:53.926Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 76 0 33 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "9790f1a2-e1dd-4fb1-950d-bbab3fe2b8dd" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.47:443 10.132.0.13:37552 - v1-completions-path 2026-10-30T03:00:59.363718Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:01:17.309700Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:01:27.574078Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:01:43.654107Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:01:46.459751Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:03:21.529026Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:03:25.027481Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:04:50.447401Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T03:04:52.854Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 73 290 154 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "e7737306-4f99-49b6-b9f3-ba07a2ce4960" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.132.0.44:8000" outbound|8000||e2e-trlp-test-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.47:42412 10.134.0.47:443 10.132.0.13:36676 - v1-completions-path 2026-10-30T03:05:05.797347Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:05:08.364174Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:05:15.822403Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:05:32.762199Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:05:35.807517Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:05:36.125202Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T03:05:36.125253Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T03:05:36.125280Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T03:05:36.125287Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T03:05:36.024Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 72 314 101 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "940435d5-d5df-420c-9da4-6bb82a90c92d" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.47:51146 10.134.0.47:443 10.132.0.13:42872 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T03:05:51.603204Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:06:02.129014Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:06:06.971884Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T03:06:10.878Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 68 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "2075ba77-1246-46eb-bcad-eeae347157dc" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.43:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.47:41902 10.134.0.47:443 10.132.0.13:41048 - v1-completions-path 2026-10-30T03:06:12.052170Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T03:06:12.052209Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T03:06:12.052225Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T03:06:12.052231Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T03:06:11.970Z] "DELETE /v1/api-keys/fd202fdd-412e-42e7-8ebd-30a652822d5c HTTP/1.1" 200 - via_upstream - "-" 0 365 81 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "99b69c59-8f87-4670-a00a-2d900116ad3a" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.132.1.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.47:58074 10.134.0.47:443 10.132.0.13:41054 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T03:06:24.162408Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:06:28.997794Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:06:32.658294Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T03:06:38.723Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 69 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "744ca732-95ed-460e-8e49-39570bf437aa" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.43:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.47:48314 10.134.0.47:443 10.132.0.13:42006 - v1-completions-path [2026-10-30T03:06:39.812Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 65 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "eada23c7-89ac-42f3-ba3f-718f37a00f20" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.43:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.47:48314 10.134.0.47:443 10.132.0.13:53026 - v1-completions-path [2026-10-30T03:06:40.194Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 62 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "e9892da3-0770-48e7-aeca-331a32b8f8f6" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.43:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.47:48314 10.134.0.47:443 10.132.0.13:53030 - v1-completions-path [2026-10-30T03:06:41.546Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 60 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "6db9579c-8662-4868-ae4c-7a26af744f56" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.43:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.47:41902 10.134.0.47:443 10.132.0.13:53052 - v1-completions-path 2026-10-30T03:06:55.160438Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:07:05.568744Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:07:08.252989Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:07:11.830152Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:07:19.571863Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T03:07:25.632Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 64 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "a7a81f13-d114-4d52-a456-831d9e53fbbd" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "10.134.0.43:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.47:48314 10.134.0.47:443 10.132.0.13:38652 - v1-completions-path [2026-10-30T03:07:25.820Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 58 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "d632084d-c366-4a6d-8255-f52f5e413793" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.47:443 10.132.0.13:38656 - v1-completions-path 2026-10-30T03:07:38.139812Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:07:48.455743Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:07:50.882408Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:07:59.324575Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:08:03.067270Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:08:06.237735Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:08:24.382843Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:08:26.812604Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:08:45.063816Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:08:55.445456Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:09:05.822594Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:09:16.112790Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:09:26.356783Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:09:36.698932Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:09:46.956514Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:09:57.339583Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:10:07.781554Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:10:17.904332Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T03:10:27.071Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 41 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "594772fa-0cd6-4d0e-afc3-07e63a54965e" "maas.apps.d9958b971f0301cd3391.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.47:443 10.132.0.13:51170 - v1-completions-path