2026-11-01T20:06:23.400022Z info FLAG: --concurrency="0" 2026-11-01T20:06:23.400050Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-01T20:06:23.400054Z info FLAG: --help="false" 2026-11-01T20:06:23.400057Z info FLAG: --log_as_json="false" 2026-11-01T20:06:23.400059Z info FLAG: --log_caller="" 2026-11-01T20:06:23.400061Z info FLAG: --log_output_level="default:info" 2026-11-01T20:06:23.400063Z info FLAG: --log_stacktrace_level="default:none" 2026-11-01T20:06:23.400067Z info FLAG: --log_target="[stdout]" 2026-11-01T20:06:23.400070Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-01T20:06:23.400072Z info FLAG: --outlierLogPath="" 2026-11-01T20:06:23.400074Z info FLAG: --profiling="true" 2026-11-01T20:06:23.400076Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-01T20:06:23.400078Z info FLAG: --proxyLogLevel="warning" 2026-11-01T20:06:23.400080Z info FLAG: --serviceCluster="istio-proxy" 2026-11-01T20:06:23.400082Z info FLAG: --stsPort="0" 2026-11-01T20:06:23.400084Z info FLAG: --templateFile="" 2026-11-01T20:06:23.400086Z info FLAG: --tokenManagerPlugin="" 2026-11-01T20:06:23.400089Z info FLAG: --vklog="0" 2026-11-01T20:06:23.400092Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-01T20:06:23.400258Z info Proxy role ips=[10.134.0.29] type=router id=maas-default-gateway-openshift-default-fd4b99879-jsxtq.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-01T20:06:23.400311Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-01T20:06:23.401433Z info cpu limit detected as 2, setting concurrency 2026-11-01T20:06:23.401636Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-01T20:06:23.401662Z info JWT policy is third-party-jwt 2026-11-01T20:06:23.401667Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-01T20:06:23.404705Z info Starting default Istio SDS Server 2026-11-01T20:06:23.404745Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-01T20:06:23.404774Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-01T20:06:23.405002Z info Opening status port 15020 2026-11-01T20:06:23.405638Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-01T20:06:23.406890Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-01T20:06:23.407768Z info Starting proxy agent 2026-11-01T20:06:23.407813Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-01T20:06:23.415514Z info sds Starting SDS grpc server 2026-11-01T20:06:23.415536Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-01T20:06:23.453985Z info cache generated new workload certificate resourceName=default latency=48.163907ms ttl=23h59m59.546017559s 2026-11-01T20:06:23.454017Z info cache Root cert has changed, start rotating root cert 2026-11-01T20:06:23.454049Z info cache returned workload trust anchor from cache ttl=23h59m59.545951824s 2026-11-01T20:06:23.477703Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-01T20:06:23.586858Z info ads ADS: new connection for node:1 2026-11-01T20:06:23.586921Z info cache returned workload certificate from cache ttl=23h59m59.413080299s 2026-11-01T20:06:23.587437Z info ads ADS: new connection for node:2 2026-11-01T20:06:23.587511Z info cache returned workload trust anchor from cache ttl=23h59m59.412490337s 2026-11-01T20:06:23.673155Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:06:32.105539Z info Readiness succeeded in 8.708457419s 2026-11-01T20:06:32.106019Z info Envoy proxy is ready 2026-11-01T20:06:32.115795Z info Readiness succeeded in 8.71872002s 2026-11-01T20:06:48.131725Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:06:54.683364Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-01T20:07:10.317Z] "POST /maas-api/internal/v1/api-keys/cleanup HTTP/1.1" 404 NR route_not_found - "-" 2 0 38 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "380df873-462b-4fd7-92fc-47d277a45554" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "-" - - 10.134.0.29:443 10.132.0.21:44486 - - 2026-11-01T20:07:10.671483Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:07:10.671604Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:07:10.671696Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T20:07:10.671757Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-01T20:07:10.649Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 91 22 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "6363eb19-ce32-4e90-a428-1d1870f42deb" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.29:53740 10.134.0.29:443 10.132.0.21:44510 - odh-ai-gateway-infra.maas-api-route.4 2026-11-01T20:08:37.031020Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:08:37.031064Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:08:37.031082Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:08:37.031089Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T20:08:37.031363Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 2026-11-01T20:08:37.031443Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-01T20:08:37.031454Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-01T20:08:37.031458Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-01T20:08:37.031463Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-11-01T20:08:36.934Z] "POST /v1/completions HTTP/1.1" 404 - via_upstream - "-" 94 126 96 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "d21ca8ad-2fed-45a8-85e3-e8654998d57c" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.132.0.60:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.29:35540 10.134.0.29:443 10.132.0.21:41288 - v1-completions-path [2026-11-01T20:08:45.329Z] "GET /v1/tenants HTTP/1.1" 404 NR route_not_found - "-" 0 0 31 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "811b3b55-92cb-40fa-a4c7-0e629de88e79" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "-" - - 10.134.0.29:443 10.132.0.21:47438 - - 2026-11-01T20:09:42.468415Z info ads ADS: new connection for node:3 2026-11-01T20:09:42.494372Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-01T20:09:42.494477Z info cache read certificate from file resource=file-root:system 2026-11-01T20:11:50.970135Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 2026-11-01T20:11:50.970187Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 2026-11-01T20:11:50.970206Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 2026-11-01T20:11:50.970213Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=27 2026-11-01T20:11:50.970491Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=27 2026-11-01T20:11:50.970614Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=27 2026-11-01T20:11:50.970667Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=27 2026-11-01T20:11:50.970702Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=27 2026-11-01T20:11:50.970726Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=27 [2026-11-01T20:11:50.856Z] "POST /v1/chat/completions HTTP/1.1" 404 - via_upstream - "-" 126 129 113 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "860837cd-0627-4ca1-a347-79ed63a34ec1" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.132.0.60:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.29:33738 10.134.0.29:443 10.132.0.21:47458 - v1-chat-completions-path 2026-11-01T20:11:52.548783Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:14:36.041223Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:14:53.577296Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:15:11.366220Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:15:27.202186Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:15:36.718695Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:15:36.718726Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:15:36.718737Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T20:15:36.718742Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:15:36.718912Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-01T20:15:36.574Z] "POST /anything/e2e-dft-75f85b-37e4d3 HTTP/1.1" 200 - via_upstream - "-" 109 1762 144 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "a4ba486c-fbf3-4edb-944d-ceeffcb524b4" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.29:44186 10.134.0.29:443 10.132.0.21:46566 - llm.external-model-e2e-dft-75f85b-80959e40cd35.0 2026-11-01T20:15:48.531773Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:15:50.937438Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:15:51.510360Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:15:52.782495Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:15:53.900180Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:15:55.989478Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:15:56.595429Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:15:59.540626Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:16:13.424475Z info ads ADS: new connection for node:4 2026-11-01T20:16:13.424796Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-01T20:16:13.424880Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-01T20:17:49.000010Z info ads ADS: "" 4 terminated 2026-11-01T20:18:10.046165Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:18:19.872294Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:18:39.801108Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:18:43.428121Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:18:43.428154Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:18:43.428165Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T20:18:43.428169Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:18:43.428330Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-01T20:18:43.179Z] "POST /anything/e2e-rt-sw-65ba84-client HTTP/1.1" 200 - via_upstream - "-" 124 1760 249 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "3e77bb09-7847-47f5-b447-340e66760e14" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.29:41936 10.134.0.29:443 10.132.0.21:54168 - llm.external-model-e2e-rt-sw-65ba84-0337161797d3.2 2026-11-01T20:18:54.684009Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:19:01.804371Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 2026-11-01T20:19:01.804421Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 2026-11-01T20:19:01.804440Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=27 2026-11-01T20:19:01.804448Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 2026-11-01T20:19:01.804859Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=27 2026-11-01T20:19:01.973921Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-01T20:19:01.979170Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:19:01.979213Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:19:01.979230Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:19:01.979236Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T20:19:01.979489Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-01T20:19:01.581Z] "POST /anything/e2e-rt-sw-65ba84-client HTTP/1.1" 200 - via_upstream - "-" 124 1752 223 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "f819917e-0642-4637-9dc1-5a599e162475" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.29:57624 10.134.0.29:443 10.132.0.21:33184 - llm.external-model-e2e-rt-sw-65ba84-0337161797d3.0 [2026-11-01T20:19:01.864Z] "POST /llm/e2e-rt-sw-65ba84/anything/e2e-rt-sw-65ba84-client HTTP/1.1" 503 UH no_healthy_upstream - "-" 124 19 114 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "6b91dfd6-8425-4934-ad04-4e07b626a8de" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rt-sw-65ba84.llm.svc.cluster.local - 10.134.0.29:443 10.132.0.21:33200 - llm.external-model-e2e-rt-sw-65ba84-0337161797d3.2 2026-11-01T20:19:17.611258Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:19:39.594809Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:20:14.407812Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:20:27.573776Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:21:38.519088Z info ads ADS: "" 3 terminated 2026-11-01T20:21:51.726975Z info ads ADS: new connection for node:5 2026-11-01T20:21:51.733783Z info cache read certificate from file resource=file-root:system 2026-11-01T20:23:15.734704Z info ads ADS: "" 5 terminated with stream closed 2026-11-01T20:25:37.375215Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:25:41.385969Z info ads ADS: new connection for node:6 2026-11-01T20:25:41.419533Z info cache read certificate from file resource=file-root:system 2026-11-01T20:26:53.351776Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:27:07.076558Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:27:51.094537Z info ads ADS: "" 6 terminated with stream closed 2026-11-01T20:29:03.110519Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:33:42.376982Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-01T20:35:16.175468Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:35:39.287889Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:37:46.365553Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:38:01.641523Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:38:03.953382Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:38:12.849935Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:38:17.961616Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:38:35.176939Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:38:54.939425Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:39:05.144368Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:39:15.526159Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:39:25.629706Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:39:35.912756Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:39:46.140076Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:39:56.352264Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:40:06.539339Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:40:16.931616Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:40:26.994115Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:41:52.530820Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-01T20:41:56.600Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 144 409 77 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "c3e6315e-f442-4208-a886-0eeadb1f54f2" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.134.0.29:55640 10.134.0.29:443 10.132.0.21:45904 - v1-chat-completions-path 2026-11-01T20:42:08.190613Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:42:09.741844Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:42:12.957004Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 2026-11-01T20:42:12.957043Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 2026-11-01T20:42:12.957058Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 2026-11-01T20:42:12.957063Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=27 [2026-11-01T20:42:12.890Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 323 66 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "ae7fb00e-72ff-41c7-bb86-0106d44f4de7" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.29:58090 10.134.0.29:443 10.132.0.21:56874 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T20:42:15.674718Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:42:19.044633Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-11-01T20:42:18.963Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 81 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "9a686bf8-883f-4b6a-a4c6-90754b6fbbb0" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.25:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.29:47062 10.134.0.29:443 10.132.0.21:56900 - v1-catch-all-path 2026-11-01T20:42:30.942879Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:42:32.353004Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:42:35.170062Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-11-01T20:42:35.083Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 87 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "d63c0394-2244-4626-9d18-647b63c38c36" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.25:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.29:47062 10.134.0.29:443 10.132.0.21:58412 - v1-catch-all-path 2026-11-01T20:42:46.579873Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:42:57.525642Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:42:59.860368Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:42:59.909256Z info ads ADS: new connection for node:7 2026-11-01T20:42:59.914634Z info cache read certificate from file resource=file-root:system 2026-11-01T20:43:00.361506Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:44:12.073336Z info ads ADS: "" 7 terminated 2026-11-01T20:44:12.148724Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:44:12.639222Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:44:15.951117Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:44:17.797998Z info ads ADS: new connection for node:8 2026-11-01T20:44:17.799041Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-01T20:44:18.637603Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:44:59.118898Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:45:17.648101Z info ads ADS: "" 8 terminated 2026-11-01T20:45:17.733114Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:45:19.326192Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:45:21.480957Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 2026-11-01T20:45:21.481002Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 2026-11-01T20:45:21.481021Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 2026-11-01T20:45:21.481028Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=27 [2026-11-01T20:45:21.408Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 90 333 72 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "c6b8ae3f-d5af-437a-98c6-2af875e72da5" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.29:53370 10.134.0.29:443 10.132.0.21:57774 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T20:45:22.316568Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:45:26.623384Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:45:38.440785Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:45:55.234944Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:46:10.882121Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:46:22.544394Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:46:26.283190Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:46:37.036054Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:46:39.486169Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:46:54.718308Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:50:10.463314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=27 2026-11-01T20:50:10.480270Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 2026-11-01T20:50:10.480317Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 2026-11-01T20:50:10.480339Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 2026-11-01T20:50:10.480347Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=27 [2026-11-01T20:50:10.407Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 88 330 72 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "88e5b8d2-ff8a-431f-a494-eb0feeb13dc2" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.29:34110 10.134.0.29:443 10.132.0.21:54162 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T20:50:54.456725Z info ads ADS: new connection for node:9 2026-11-01T20:50:54.468324Z info cache read certificate from file resource=file-root:system 2026-11-01T20:50:54.973528Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:50:56.400260Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:51:08.389139Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:51:08.389182Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:51:08.389197Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:51:08.389203Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T20:51:08.389401Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-01T20:51:08.257Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1713 132 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "d2b166fa-c006-4513-aadb-25fc9ad8c8f3" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.29:35286 10.134.0.29:443 10.132.0.21:38112 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-01T20:51:14.846563Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:51:14.846595Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:51:14.846606Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:51:14.846611Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T20:51:14.846819Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-01T20:51:14.751Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1711 94 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "59005b6b-2b15-42c9-9e62-99234cf5ed37" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.29:35286 10.134.0.29:443 10.132.0.21:56080 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-01T20:51:21.416857Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 2026-11-01T20:51:21.416896Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 2026-11-01T20:51:21.416910Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=27 2026-11-01T20:51:21.416916Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 2026-11-01T20:51:21.417106Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=27 [2026-11-01T20:51:21.292Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1716 124 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "463f26ea-3820-4fb6-83b5-dffcf242e122" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.29:41238 10.134.0.29:443 10.132.0.21:46786 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-01T20:52:28.573412Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:52:28.722314Z info ads ADS: "" 9 terminated with stream closed 2026-11-01T20:52:29.116446Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:52:32.403984Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-01T20:52:35.287Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 93 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "e797278b-3490-4202-b954-a20570742490" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.29:49314 10.134.0.29:443 10.132.0.21:56912 - v1-completions-path [2026-11-01T20:52:47.534Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 63 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "43974572-9c16-4c04-8ebf-6effb82e9596" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.132.0.60:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.29:35540 10.134.0.29:443 10.132.0.21:47488 - v1-completions-path 2026-11-01T20:52:48.910635Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:52:52.176818Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:53:00.648837Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-01T20:53:06.291Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 61 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "39667d03-8e93-4ee1-b8c4-6b101e527587" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.29:40304 10.134.0.29:443 10.132.0.21:52550 - v1-completions-path 2026-11-01T20:53:21.962142Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:53:32.193110Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:53:33.671556Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:53:39.142901Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:53:39.142938Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T20:53:39.142954Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T20:53:39.142959Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-01T20:53:39.072Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 318 70 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "65214072-bbe2-4ff7-b9e6-e08fe0d6139c" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.29:44680 10.134.0.29:443 10.132.0.21:55956 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T20:53:40.142Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 56 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "f8c95d1c-e11d-4a90-8e77-3bbbe0662495" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.29:49314 10.134.0.29:443 10.132.0.21:34276 - v1-completions-path 2026-11-01T20:53:52.551930Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:53:54.044318Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:53:59.367526Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:54:10.236518Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:54:23.916477Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:54:25.345756Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:54:30.812690Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:54:44.810609Z info cache event for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt : REMOVE, pushing to proxy 2026-11-01T20:54:49.054632Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:54:51.773412Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:55:05.503034Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:55:07.022811Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:55:12.339664Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:55:27.408657Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:55:37.613255Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:55:52.646810Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:55:55.055678Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:58:07.530724Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:58:10.020146Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:58:12.952968Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-01T20:58:16.242Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 68 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "254ac898-5e53-4e22-aa23-7485443a7b07" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.132.0.60:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.29:33738 10.134.0.29:443 10.132.0.21:54282 - v1-completions-path 2026-11-01T20:58:28.006354Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:58:37.056920Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:58:41.123852Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-01T20:58:44.296Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 60 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "12c1a345-488b-4e95-9759-df7e41c41de5" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.29:40304 10.134.0.29:443 10.132.0.21:35788 - v1-completions-path [2026-11-01T20:58:44.938Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 52 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "329c2bca-2e8d-4820-8936-76d3695a8bfa" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.29:443 10.132.0.21:35796 - v1-completions-path 2026-11-01T20:58:56.722353Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:59:07.085196Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:59:09.981968Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:59:13.497906Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 2026-11-01T20:59:13.497944Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 2026-11-01T20:59:13.497960Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=27 2026-11-01T20:59:13.497965Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 [2026-11-01T20:59:13.432Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 67 310 65 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "65be613f-a32a-4754-8948-7e56fa4a4a61" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.133.0.130:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.29:48742 10.134.0.29:443 10.132.0.21:52798 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T20:59:14.174Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 58 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "94d3cf8e-0eec-4b7a-b3d9-ea9fafbbfa00" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.29:49314 10.134.0.29:443 10.132.0.21:52806 - v1-completions-path [2026-11-01T20:59:14.815Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 56 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "861d6296-3fde-49bf-a3c9-1198098319ab" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.29:443 10.132.0.21:52814 - v1-completions-path 2026-11-01T20:59:26.575693Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:59:28.523187Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:59:31.832979Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T20:59:36.300239Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 2026-11-01T20:59:36.300274Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 2026-11-01T20:59:36.300289Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=27 2026-11-01T20:59:36.300294Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=27 [2026-11-01T20:59:36.241Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 68 311 58 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "9be051e3-31f2-46ea-9b51-b48cc6220eae" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.133.0.130:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.29:48742 10.134.0.29:443 10.132.0.21:35398 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T20:59:53.293580Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T21:00:03.586045Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T21:00:05.526968Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T21:00:08.869844Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T21:00:14.645861Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-01T21:00:19.833Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 68 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "a17c8bfc-fb73-4415-baa9-9b66f893bd87" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.29:40304 10.134.0.29:443 10.132.0.21:60332 - v1-completions-path [2026-11-01T21:00:20.725Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 60 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "7544cad3-1af0-4bce-8290-7ab4b35643c2" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.29:49314 10.134.0.29:443 10.132.0.21:60334 - v1-completions-path [2026-11-01T21:00:21.157Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 55 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "c2eb3a0a-2e54-4f4b-9e81-44997e67104e" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.29:49314 10.134.0.29:443 10.132.0.21:60342 - v1-completions-path 2026-11-01T21:00:33.161680Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T21:00:43.405540Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-01T21:00:48.289Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 288 73 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "f9806851-97a7-42a9-ba50-20d70c142c43" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.132.0.60:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.29:35540 10.134.0.29:443 10.132.0.21:42736 - v1-completions-path [2026-11-01T21:00:48.423Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 289 74 - "44.212.242.249,10.132.0.21" "python-requests/2.33.0" "5183345e-9c5e-47b7-b9d0-94fa3607e594" "maas.apps.4ec91afccb26205cb961.prod.konflux-ocp-ci.dev" "10.132.0.60:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.29:35540 10.134.0.29:443 10.132.0.21:42744 - v1-completions-path