2026-11-03T20:11:44.047274Z info FLAG: --concurrency="0" 2026-11-03T20:11:44.047292Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T20:11:44.047296Z info FLAG: --help="false" 2026-11-03T20:11:44.047299Z info FLAG: --log_as_json="false" 2026-11-03T20:11:44.047301Z info FLAG: --log_caller="" 2026-11-03T20:11:44.047303Z info FLAG: --log_output_level="default:info" 2026-11-03T20:11:44.047305Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T20:11:44.047309Z info FLAG: --log_target="[stdout]" 2026-11-03T20:11:44.047311Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T20:11:44.047313Z info FLAG: --outlierLogPath="" 2026-11-03T20:11:44.047315Z info FLAG: --profiling="true" 2026-11-03T20:11:44.047317Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T20:11:44.047319Z info FLAG: --proxyLogLevel="warning" 2026-11-03T20:11:44.047321Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T20:11:44.047323Z info FLAG: --stsPort="0" 2026-11-03T20:11:44.047325Z info FLAG: --templateFile="" 2026-11-03T20:11:44.047327Z info FLAG: --tokenManagerPlugin="" 2026-11-03T20:11:44.047329Z info FLAG: --vklog="0" 2026-11-03T20:11:44.047332Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T20:11:44.047559Z info Proxy role ips=[10.133.0.40] type=router id=maas-default-gateway-openshift-default-59f957fdd7-jbqn5.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T20:11:44.047614Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T20:11:44.048863Z info cpu limit detected as 2, setting concurrency 2026-11-03T20:11:44.049050Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T20:11:44.049061Z info JWT policy is third-party-jwt 2026-11-03T20:11:44.049064Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T20:11:44.050691Z info Starting default Istio SDS Server 2026-11-03T20:11:44.050723Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T20:11:44.050748Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T20:11:44.051334Z info Opening status port 15020 2026-11-03T20:11:44.052394Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T20:11:44.052782Z info sds Starting SDS grpc server 2026-11-03T20:11:44.052812Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T20:11:44.053873Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T20:11:44.054671Z info Starting proxy agent 2026-11-03T20:11:44.054700Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T20:11:44.088121Z info cache generated new workload certificate resourceName=default latency=35.208462ms ttl=23h59m59.911885541s 2026-11-03T20:11:44.088165Z info cache Root cert has changed, start rotating root cert 2026-11-03T20:11:44.088208Z info cache returned workload trust anchor from cache ttl=23h59m59.911793094s 2026-11-03T20:11:44.119918Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T20:11:44.220838Z info ads ADS: new connection for node:1 2026-11-03T20:11:44.220928Z info cache returned workload certificate from cache ttl=23h59m59.779073768s 2026-11-03T20:11:44.221636Z info ads ADS: new connection for node:2 2026-11-03T20:11:44.221714Z info cache returned workload trust anchor from cache ttl=23h59m59.778288489s 2026-11-03T20:11:44.313623Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:11:52.758208Z info Readiness succeeded in 8.713414635s 2026-11-03T20:11:52.758668Z info Envoy proxy is ready 2026-11-03T20:14:16.964880Z info ads ADS: new connection for node:3 2026-11-03T20:14:16.983091Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-03T20:14:16.983157Z info cache read certificate from file resource=file-root:system 2026-11-03T20:14:17.677277Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:14:29.338956Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:14:29.719755Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T20:14:29.719786Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T20:14:29.719797Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T20:14:29.719802Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T20:14:29.598Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 74 315 121 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "24ff32a4-7171-4ab4-87f4-5d6229723fe7" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.40:49338 10.133.0.40:443 10.132.0.13:51090 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T20:14:37.053049Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:14:58.434180Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T20:14:58.434221Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T20:14:58.434236Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T20:14:58.434242Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T20:14:58.434482Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 2026-11-03T20:14:58.620808Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T20:14:58.620859Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T20:14:58.620881Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T20:14:58.620888Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T20:14:58.621191Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T20:14:58.297Z] "POST /anything/e2e-rt-sw-9dcd2a-client HTTP/1.1" 200 - via_upstream - "-" 124 1577 136 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "6e8f72d9-6f83-4905-b789-a230df30f30d" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.40:39100 10.133.0.40:443 10.132.0.13:46104 - llm.external-model-e2e-rt-sw-9dcd2a-914404c28491.0 [2026-11-03T20:14:58.493Z] "POST /anything/e2e-rt-sw-9dcd2a-client HTTP/1.1" 200 - via_upstream - "-" 124 1577 127 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "e806c7ba-8f0c-491d-84f1-a6d6e285407d" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.40:39108 10.133.0.40:443 10.132.0.13:46120 - llm.external-model-e2e-rt-sw-9dcd2a-914404c28491.0 2026-11-03T20:14:59.708113Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T20:14:59.708144Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T20:14:59.708154Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T20:14:59.708158Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T20:14:59.708303Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T20:14:59.601Z] "POST /llm/e2e-rt-sw-9dcd2a/anything/e2e-rt-sw-9dcd2a-client HTTP/1.1" 503 UH no_healthy_upstream - "-" 124 19 106 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "f1a0b3fa-03be-4a37-b816-397e900dddc7" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rt-sw-9dcd2a.llm.svc.cluster.local - 10.133.0.40:443 10.132.0.13:46130 - llm.external-model-e2e-rt-sw-9dcd2a-914404c28491.2 2026-11-03T20:15:05.900080Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:15:36.582854Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:15:53.217689Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:16:22.443510Z info ads ADS: "" 3 terminated 2026-11-03T20:16:22.821380Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:16:26.091863Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:16:49.166219Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:17:10.553149Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:17:32.863405Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:18:38.228122Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T20:18:38.228155Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T20:18:38.228167Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T20:18:38.228172Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T20:18:38.080Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 147 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "9cdea92c-1a77-46bf-b86c-8fc80cf6d55b" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.40:59400 10.133.0.40:443 10.132.0.13:58892 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T20:18:39.374Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 139 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "8328ca17-2e17-4c3b-bb71-63f4bd5eeb90" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.133.0.40:443 10.132.0.13:58900 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T20:18:39.768723Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T20:18:39.768774Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T20:18:39.768791Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T20:18:39.768798Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T20:18:40.138101Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-03T20:18:40.141400Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T20:18:40.141427Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T20:18:40.141467Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T20:18:40.141474Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T20:18:39.708Z] "DELETE /v1/api-keys/1636f25f-3879-4e79-9dd6-5d43499d2878 HTTP/1.1" 200 - via_upstream - "-" 0 321 60 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "8b4b4e80-146f-4b99-885a-9ffe1ca4d7c1" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.40:59412 10.133.0.40:443 10.132.0.13:58914 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T20:18:39.984Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 156 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "71cd684f-7e0c-4398-aee5-8c8c266e3f91" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.40:59400 10.133.0.40:443 10.132.0.13:58928 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T20:19:14.529977Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T20:19:41.144Z] "POST /maas-api/internal/v1/api-keys/cleanup HTTP/1.1" 404 NR route_not_found - "-" 2 0 13 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "dfc90189-f742-4757-a8f3-85a71887b85d" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "-" - - 10.133.0.40:443 10.132.0.13:53810 - - 2026-11-03T20:19:43.608984Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T20:19:43.609104Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T20:19:43.609161Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T20:19:43.609194Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T20:19:43.594Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 91 14 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "a22652b6-f843-463c-9661-9f200a02826c" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.40:59328 10.133.0.40:443 10.132.0.13:53816 - odh-ai-gateway-infra.maas-api-route.4 [2026-11-03T20:23:26.851Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 110 331 81 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "350dd504-4bdc-4388-b28d-ed2c31c0fa5d" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "10.134.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.40:41524 10.133.0.40:443 10.132.0.13:52380 - v1-chat-completions-path [2026-11-03T20:24:40.485Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 113 328 71 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "17f7c2b5-d28b-403f-bc2e-0392fbb35c00" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "10.134.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.40:58802 10.133.0.40:443 10.132.0.13:48890 - v1-chat-completions-path 2026-11-03T20:33:14.522841Z info ads ADS: new connection for node:4 2026-11-03T20:33:14.528440Z info cache read certificate from file resource=file-root:system 2026-11-03T20:39:20.961288Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-03T20:40:06.435614Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:40:43.412860Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:41:07.125084Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:41:09.688931Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:41:10.343969Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:41:16.746865Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:41:22.888220Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:41:54.911963Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:41:55.779337Z info ads ADS: new connection for node:5 2026-11-03T20:41:55.779556Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T20:41:55.779586Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T20:41:59.859059Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:42:36.997581Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:43:13.886372Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:43:35.163786Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:43:47.655903Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:43:48.473794Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:43:55.482678Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:46:45.940543Z info ads ADS: "" 5 terminated 2026-11-03T20:51:11.159389Z info ads ADS: "" 4 terminated with stream closed 2026-11-03T20:52:08.680901Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:53:47.905613Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:54:58.704381Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:56:35.937635Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:57:09.813013Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=3 2026-11-03T21:00:53.700421Z info cache event for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt : REMOVE, pushing to proxy 2026-11-03T21:02:55.424623Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=4 2026-11-03T21:06:46.644244Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:06:46.644279Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:06:46.644294Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T21:06:46.644299Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T21:06:46.564Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 30 315 79 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "acd5c918-f026-4368-b176-c92faf6a1ae4" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.40:42240 10.133.0.40:443 10.132.0.13:41228 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:06:48.821976Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:07:12.011837Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:07:46.260937Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:08:02.256255Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:08:04.128791Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:08:10.438641Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:08:13.823467Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-11-03T21:08:13.842165Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-11-03T21:08:13.761Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 80 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "5dc5f991-202a-4ee4-8c1c-506c0fae2dbe" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "10.133.0.35:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.40:36274 10.133.0.40:443 10.132.0.13:36036 - v1-catch-all-path 2026-11-03T21:08:26.148536Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:08:27.962412Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:08:42.911994Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:08:53.813569Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:08:55.895379Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:08:55.940727Z info ads ADS: new connection for node:6 2026-11-03T21:08:55.940981Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T21:08:55.941014Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T21:08:56.858991Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:09:11.713152Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:09:11.739267Z info ads ADS: "" 6 terminated 2026-11-03T21:09:12.142784Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:09:15.258463Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:09:15.976386Z info ads ADS: new connection for node:7 2026-11-03T21:09:15.976668Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T21:09:17.893837Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:09:40.774811Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=5 2026-11-03T21:09:56.608661Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:10:02.591Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 336 85 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "c4de1830-d723-42fb-9468-13c9bc4372f5" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "10.133.0.188:8000" outbound|54321||e2e-prio-pool-0704dddb-inference-pool-ip-461f2a7f.llm.svc.cluster.local 10.133.0.40:42748 10.133.0.40:443 10.132.0.13:35154 - v1-model-routing 2026-11-03T21:10:16.046187Z info ads ADS: "" 7 terminated with stream closed 2026-11-03T21:10:16.114526Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:10:18.064991Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:10:20.747378Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:10:25.455050Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:12:33.781553Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:12:33.781589Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:12:33.781604Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T21:12:33.781610Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T21:12:33.708Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 33 325 73 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "be6c5865-25e7-463a-8ded-3ba4fe10079c" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.40:59172 10.133.0.40:443 10.132.0.13:39888 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:12:39.193182Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:12:55.072795Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:13:09.167117Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:13:09.167153Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:13:09.167167Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:13:09.167172Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T21:13:09.111Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 628 55 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "0cffc8a9-f270-42d9-8575-82fb2fab7a9a" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.40:46278 10.133.0.40:443 10.132.0.13:42410 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:13:10.707464Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:13:22.434468Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:13:26.516096Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:13:37.814153Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:13:40.455525Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:13:55.924697Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:14:40.557545Z info ads ADS: new connection for node:8 2026-11-03T21:14:40.563496Z info cache read certificate from file resource=file-root:system 2026-11-03T21:14:41.077568Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:14:42.927423Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:15:32.238101Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:15:32.238134Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:15:32.238145Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T21:15:32.238150Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:15:32.238302Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T21:15:32.109Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1535 128 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "40a87bfd-3077-4dd6-81be-fb08190fc74c" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.40:34162 10.133.0.40:443 10.132.0.13:35050 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T21:15:45.235606Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:15:45.235644Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:15:45.235662Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T21:15:45.235668Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:15:45.235895Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T21:15:45.139Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1533 96 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "3b471ec9-ee0d-4bf3-b6a7-29e1361cc003" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.40:34162 10.133.0.40:443 10.132.0.13:38934 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T21:16:04.714803Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:16:04.714854Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:16:04.714876Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:16:04.714884Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T21:16:04.715109Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T21:16:04.616Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1527 98 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "dfd1a342-1daa-4abf-9b37-95a743388d46" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.40:34162 10.133.0.40:443 10.132.0.13:49316 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T21:16:17.906210Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:16:18.054132Z info ads ADS: "" 8 terminated with stream closed 2026-11-03T21:16:18.348321Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:16:18.348357Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:16:18.348369Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:16:18.348374Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T21:16:18.433969Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T21:16:18.433994Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T21:16:18.433998Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T21:16:18.434002Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 2026-11-03T21:16:18.726238Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:16:18.283Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 74 316 65 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "2a4fbc03-bf23-4b42-a4b7-691d99f4996c" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.40:60610 10.133.0.40:443 10.132.0.13:38156 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T21:16:18.399Z] "POST /llm/premium-simulated-simulated-premium/v1/completions HTTP/1.1" 403 - - "-" 74 77 34 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "0d12af92-cc4b-49ae-8a36-74c6efe6d697" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "-" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.40:443 10.132.0.13:38160 - v1-completions-path 2026-11-03T21:16:21.683497Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:16:25.362396Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:16:36.762347Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:16:38.703763Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:16:54.041203Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:16:55.917766Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:16:59.159410Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:17:06.853924Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:17:13.483Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 58 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "bc357dd2-24b7-4c13-b820-2dfa35046fba" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.40:443 10.132.0.13:34778 - v1-completions-path [2026-11-03T21:17:14.305Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 79 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "17248516-5156-40ea-a88c-f2370720dce1" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.40:47556 10.133.0.40:443 10.132.0.13:40378 - v1-completions-path [2026-11-03T21:17:15.037Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 55 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "861607af-cdb7-4558-acb5-18214db3de80" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.40:47556 10.133.0.40:443 10.132.0.13:40402 - v1-completions-path [2026-11-03T21:17:15.377Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 54 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "d0177717-4220-4d2e-804d-fc41db79023b" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.40:47556 10.133.0.40:443 10.132.0.13:40414 - v1-completions-path 2026-11-03T21:17:30.988987Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:17:41.262690Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:17:43.227632Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:18:02.083569Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:18:04.058254Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:18:10.135458Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:18:21.026131Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:18:34.692438Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:18:36.517910Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:18:42.503816Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:19:00.111946Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:19:03.019056Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:19:18.756395Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:19:21.234575Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:19:21.048Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 66 18 61 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "789b2388-5d1c-4cb1-8ed4-5eb9c1df8b9d" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.40:443 10.132.0.13:47452 - v1-completions-path 2026-11-03T21:19:28.550107Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:19:46.840990Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:19:57.121026Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:20:13.472897Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:20:16.639224Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:24:04.053425Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:24:07.539289Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:24:07.539320Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:24:07.539329Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:24:07.539333Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T21:24:06.931Z] "DELETE /v1/api-keys/e2e-trlp-test-key HTTP/1.1" 404 - via_upstream - "-" 0 29 608 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "d1a74f68-f2ae-4506-a272-6531cc016df6" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.40:32976 10.133.0.40:443 10.132.0.13:53748 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:24:19.198633Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:24:21.052842Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:24:28.278761Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:24:46.477651Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:24:50.236999Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:25:05.967417Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:25:16.212390Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:25:18.721183Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:25:26.051Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 65 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "5f5b8e05-477c-4c43-b3ee-d3f18370c714" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.40:47556 10.133.0.40:443 10.132.0.13:44824 - v1-completions-path 2026-11-03T21:25:39.180684Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:25:41.596139Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:25:45.595096Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:25:52.950Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 68 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "319c223d-4906-4961-a3d9-fe7c57ea8a44" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.40:59998 10.133.0.40:443 10.132.0.13:45456 - v1-completions-path [2026-11-03T21:25:54.033Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 64 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "5a818f3e-7a3a-48b1-84d6-61e250ed3348" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.40:47556 10.133.0.40:443 10.132.0.13:32820 - v1-completions-path [2026-11-03T21:25:54.248Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 67 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "64d7b52b-4d02-42a4-8241-469ce634aea1" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.40:59998 10.133.0.40:443 10.132.0.13:32834 - v1-completions-path 2026-11-03T21:26:06.601510Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:26:16.818667Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:26:19.229267Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:26:23.183193Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:26:28.612973Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:26:35.096Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 63 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "b2938cb4-580f-4419-ad50-73b8b3cd333d" "maas.apps.96089cf8aa8b05683edc.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.40:59998 10.133.0.40:443 10.132.0.13:46606 - v1-completions-path 2026-11-03T21:26:47.202160Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:26:57.340102Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:26:59.235901Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:27:09.208655Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:27:13.092707Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:27:17.874628Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:27:35.293787Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:27:55.080793Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:28:05.378381Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:28:15.485491Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:28:25.732116Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:28:35.933035Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:28:46.234128Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:28:56.316812Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:29:06.522542Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:29:16.719717Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:29:26.956240Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8