2026-11-03T13:59:51.958527Z info FLAG: --concurrency="0" 2026-11-03T13:59:51.958546Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T13:59:51.958550Z info FLAG: --help="false" 2026-11-03T13:59:51.958553Z info FLAG: --log_as_json="false" 2026-11-03T13:59:51.958555Z info FLAG: --log_caller="" 2026-11-03T13:59:51.958557Z info FLAG: --log_output_level="default:info" 2026-11-03T13:59:51.958559Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T13:59:51.958563Z info FLAG: --log_target="[stdout]" 2026-11-03T13:59:51.958565Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T13:59:51.958568Z info FLAG: --outlierLogPath="" 2026-11-03T13:59:51.958570Z info FLAG: --profiling="true" 2026-11-03T13:59:51.958572Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T13:59:51.958574Z info FLAG: --proxyLogLevel="warning" 2026-11-03T13:59:51.958575Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T13:59:51.958577Z info FLAG: --stsPort="0" 2026-11-03T13:59:51.958579Z info FLAG: --templateFile="" 2026-11-03T13:59:51.958581Z info FLAG: --tokenManagerPlugin="" 2026-11-03T13:59:51.958584Z info FLAG: --vklog="0" 2026-11-03T13:59:51.958586Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T13:59:51.958733Z info Proxy role ips=[10.132.0.58] type=router id=maas-default-gateway-openshift-default-76f4876667-2bd6m.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T13:59:51.958782Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T13:59:51.960214Z info cpu limit detected as 2, setting concurrency 2026-11-03T13:59:51.960405Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T13:59:51.960415Z info JWT policy is third-party-jwt 2026-11-03T13:59:51.960418Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T13:59:51.962345Z info Starting default Istio SDS Server 2026-11-03T13:59:51.962375Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T13:59:51.962401Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T13:59:51.963061Z info Opening status port 15020 2026-11-03T13:59:51.963447Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T13:59:51.964758Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T13:59:51.965667Z info Starting proxy agent 2026-11-03T13:59:51.965699Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T13:59:51.974382Z info sds Starting SDS grpc server 2026-11-03T13:59:51.974401Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T13:59:52.033151Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T13:59:52.110539Z info cache generated new workload certificate resourceName=default latency=147.219992ms ttl=23h59m59.889465979s 2026-11-03T13:59:52.110590Z info cache Root cert has changed, start rotating root cert 2026-11-03T13:59:52.110633Z info cache returned workload trust anchor from cache ttl=23h59m59.889367807s 2026-11-03T13:59:52.142296Z info ads ADS: new connection for node:1 2026-11-03T13:59:52.142377Z info cache returned workload certificate from cache ttl=23h59m59.857625591s 2026-11-03T13:59:52.142811Z info ads ADS: new connection for node:2 2026-11-03T13:59:52.142908Z info cache returned workload trust anchor from cache ttl=23h59m59.857093631s 2026-11-03T13:59:52.227333Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:00:00.249540Z info Readiness succeeded in 8.293447642s 2026-11-03T14:00:00.249976Z info Envoy proxy is ready 2026-11-03T14:00:04.396675Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:00:04.396729Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:00:04.396757Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T14:00:04.396767Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T14:00:04.301Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 95 - "44.212.242.249,10.132.0.13" "curl/7.76.1" "a6b91a98-be7d-424b-a762-ac9097e216b0" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.132.0.48:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.58:43290 10.132.0.58:443 10.132.0.13:49390 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T14:00:04.484Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 357 75 - "44.212.242.249,10.132.0.13" "curl/7.76.1" "b18cde04-de76-4c41-8a45-37b9f39b22cb" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.58:45436 10.132.0.58:443 10.132.0.13:49416 - v1-model-routing [2026-11-03T14:00:04.643Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 345 60 - "44.212.242.249,10.132.0.13" "curl/7.76.1" "1f4c89c6-1667-4c5c-8568-182e5d1ad687" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.58:45442 10.132.0.58:443 10.132.0.13:49422 - v1-model-routing [2026-11-03T14:00:04.771Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 420 57 - "44.212.242.249,10.132.0.13" "curl/7.76.1" "243ff24a-cf34-465d-b090-250b9d089714" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.58:45436 10.132.0.58:443 10.132.0.13:49424 - v1-model-routing [2026-11-03T14:00:04.890Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 377 54 - "44.212.242.249,10.132.0.13" "curl/7.76.1" "a37af4bf-f9b2-4bba-a102-6360563585e6" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.58:45436 10.132.0.58:443 10.132.0.13:49440 - v1-model-routing [2026-11-03T14:00:05.003Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 338 59 - "44.212.242.249,10.132.0.13" "curl/7.76.1" "acb27c19-ae3b-4da1-b854-53a6c6761b6a" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.58:45442 10.132.0.58:443 10.132.0.13:49454 - v1-model-routing [2026-11-03T14:00:05.246Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 54 - "44.212.242.249,10.132.0.13" "curl/7.76.1" "6dba52e4-8f4d-44e6-86d8-2a1390571317" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.58:443 10.132.0.13:49456 - v1-model-routing [2026-11-03T14:00:05.360Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 53 - "44.212.242.249,10.132.0.13" "curl/7.76.1" "a89bfac8-b33e-4956-a3e5-1baa9bd797ca" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.58:443 10.132.0.13:49458 - v1-model-routing 2026-11-03T14:00:06.501515Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-11-03T14:00:06.552900Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:00:06.552937Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:00:06.552952Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:00:06.552983Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T14:00:06.494Z] "GET /v1/subscriptions HTTP/1.1" 200 - via_upstream - "-" 0 488 58 - "44.212.242.249,10.132.0.13" "curl/7.76.1" "dee9f9a0-140a-48c1-8c33-8f7807d82ea2" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.132.0.48:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.58:43290 10.132.0.58:443 10.132.0.13:49460 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T14:02:52.740524Z info ads ADS: new connection for node:3 2026-11-03T14:02:52.757849Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-03T14:02:52.757916Z info cache read certificate from file resource=file-root:system 2026-11-03T14:02:53.361357Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:03:01.506925Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:03:12.216813Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:03:17.051246Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:03:17.051302Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:03:17.051325Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:03:17.051333Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T14:03:17.051653Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T14:03:16.748Z] "POST /anything/e2e-rt-sw-14717f-client HTTP/1.1" 200 - via_upstream - "-" 124 1761 303 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "3367ea57-83d5-49bc-807a-4ab2f7073923" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.58:33452 10.132.0.58:443 10.132.0.13:43460 - llm.external-model-e2e-rt-sw-14717f-d8bcbdf290bf.0 2026-11-03T14:03:22.252527Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:03:40.119624Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:03:40.119664Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:03:40.119675Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:03:40.119679Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T14:03:40.119857Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T14:03:39.964Z] "POST /anything/e2e-rt-sw-14717f-client HTTP/1.1" 200 - via_upstream - "-" 124 1763 155 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "6925fe73-e807-48f6-9681-48208e0653ca" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.58:49006 10.132.0.58:443 10.132.0.13:53222 - llm.external-model-e2e-rt-sw-14717f-d8bcbdf290bf.0 2026-11-03T14:03:45.956836Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:03:45.956875Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:03:45.956889Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:03:45.956895Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T14:03:45.957090Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T14:03:45.875Z] "POST /llm/e2e-rt-sw-14717f/anything/e2e-rt-sw-14717f-client HTTP/1.1" 503 UH no_healthy_upstream - "-" 124 19 81 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "55618edf-e50e-47aa-9292-a8dd9cc3f190" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rt-sw-14717f.llm.svc.cluster.local - 10.132.0.58:443 10.132.0.13:49476 - llm.external-model-e2e-rt-sw-14717f-d8bcbdf290bf.2 2026-11-03T14:03:48.050312Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:03:51.748254Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:03:51.748291Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:03:51.748305Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T14:03:51.748310Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:03:51.748481Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T14:03:51.673Z] "POST /llm/e2e-rt-sw-14717f/anything/e2e-rt-sw-14717f-client HTTP/1.1" 503 UH no_healthy_upstream - "-" 124 19 74 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "51143cd6-4476-4a20-aae3-3d877a0fe8d6" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rt-sw-14717f.llm.svc.cluster.local - 10.132.0.58:443 10.132.0.13:49490 - llm.external-model-e2e-rt-sw-14717f-d8bcbdf290bf.2 2026-11-03T14:04:08.025703Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:04:08.025733Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:04:08.025744Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:04:08.025748Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T14:04:08.025921Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T14:04:07.886Z] "POST /anything/e2e-rt-sw-14717f-client HTTP/1.1" 200 - via_upstream - "-" 124 1764 139 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "7d8f07c8-8e54-4078-872f-182facba884a" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.58:40428 10.132.0.58:443 10.132.0.13:34656 - llm.external-model-e2e-rt-sw-14717f-d8bcbdf290bf.0 2026-11-03T14:04:19.603735Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:04:35.914139Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:04:46.226283Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:04:46.226334Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:04:46.226359Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T14:04:46.226367Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T14:04:46.081Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 74 317 144 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "a80194a2-fdd9-4175-93df-096b5f681896" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.132.0.48:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.58:49386 10.132.0.58:443 10.132.0.13:42840 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T14:04:47.683584Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-11-03T14:04:47.894000Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:04:47.894039Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:04:47.894056Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T14:04:47.894062Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:04:47.894322Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T14:04:47.676Z] "POST /anything/e2e-rt-eq-c1150b-client HTTP/1.1" 200 - via_upstream - "-" 124 1765 218 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "ed028135-4581-41df-b772-bf4236845f85" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.58:57908 10.132.0.58:443 10.132.0.13:42844 - llm.external-model-e2e-rt-eq-c1150b-d37cbf613683.0 2026-11-03T14:05:09.062518Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:05:17.766725Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:05:17.801065Z info ads ADS: "" 3 terminated with stream closed 2026-11-03T14:05:29.072877Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:06:13.359691Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:07:08.404456Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:08:04.850686Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:08:27.764756Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:08:27.764791Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:08:27.764815Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T14:08:27.764820Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T14:08:27.609Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 155 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "a2af0835-4cbb-4c6b-8eaa-225ce9daadf6" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.132.0.48:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.58:39544 10.132.0.58:443 10.132.0.13:32974 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T14:08:28.592318Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:08:28.592353Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:08:28.592368Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T14:08:28.592372Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:08:28.955617Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:08:28.955654Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:08:28.955671Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:08:28.955677Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T14:08:29.170485Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:08:29.170531Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:08:29.170557Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:08:29.170566Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T14:08:28.529Z] "DELETE /v1/api-keys/b2cb96b6-1841-4fa6-b579-f070802e0b66 HTTP/1.1" 200 - via_upstream - "-" 0 321 62 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "9cc99291-e6f2-4b87-b254-daa318343d70" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.132.0.48:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.58:39554 10.132.0.58:443 10.132.0.13:32984 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T14:08:28.887Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 68 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "da8ce234-2c72-4a9d-9120-5f85433ad638" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.132.0.48:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.58:39554 10.132.0.58:443 10.132.0.13:32996 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T14:08:29.014Z] "DELETE /v1/api-keys/2e3cde0a-c973-475d-952a-367ab9f0b311 HTTP/1.1" 200 - via_upstream - "-" 0 321 155 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "788cf923-aeb8-4a2a-a8f9-e45f4a34978f" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.132.0.48:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.58:39544 10.132.0.58:443 10.132.0.13:33002 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T14:08:29.475430Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-03T14:08:29.475528Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:08:29.475562Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:08:29.475591Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:08:29.475603Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T14:08:29.413Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 62 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "fbf9284e-b95a-4a6d-8d6e-3e400f508ff5" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.132.0.48:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.58:39544 10.132.0.58:443 10.132.0.13:33010 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T14:08:37.828Z] "POST /maas-api/internal/v1/subscriptions/select HTTP/1.1" 404 NR route_not_found - "-" 56 0 7 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "4a29ee38-66f6-4f3d-9b3b-3ee6d35c4322" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "-" - - 10.132.0.58:443 10.132.0.13:57062 - - 2026-11-03T14:08:38.722660Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:08:38.722698Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:08:38.722716Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:08:38.722720Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T14:08:38.665Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 57 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "23747ea3-d6fc-4794-b4c0-378e7c55b557" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.132.0.48:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.58:39544 10.132.0.58:443 10.132.0.13:57068 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T14:08:49.956685Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:09:08.521092Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:22:34.870127Z info ads ADS: new connection for node:4 2026-11-03T14:22:34.875883Z info cache read certificate from file resource=file-root:system 2026-11-03T14:28:42.164024Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-03T14:30:19.969488Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:30:45.083285Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:31:08.410834Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:31:26.785642Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:32:22.985176Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:32:27.077638Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:32:27.077684Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:32:27.077708Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:32:27.077717Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T14:32:26.965Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 66 308 112 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "d73762bd-a624-4eb5-a775-a6c7dfd0c53a" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.132.0.48:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.58:33960 10.132.0.58:443 10.132.0.13:47036 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T14:32:41.352701Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:32:45.234221Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:32:50.413721Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:32:50.861257Z info ads ADS: new connection for node:5 2026-11-03T14:32:50.861615Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T14:32:50.861720Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T14:32:52.901069Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:32:58.559674Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:33:18.375647Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:36:53.076624Z info ads ADS: "" 5 terminated with stream closed 2026-11-03T14:38:01.346913Z info ads ADS: "" 4 terminated 2026-11-03T14:41:03.696574Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:42:13.088737Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:44:16.335661Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:44:53.785094Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:46:05.910547Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:49:08.351254Z info cache event for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt : REMOVE, pushing to proxy 2026-11-03T14:50:51.388747Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=3 2026-11-03T14:55:49.646728Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:56:13.361286Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:56:24.996433Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:56:24.996467Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:56:24.996479Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:56:24.996483Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T14:56:24.937Z] "POST /v1/api-keys HTTP/1.1" 400 - via_upstream - "-" 80 94 59 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "455bf86b-4ce5-4c33-8e82-032d85c528e8" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.132.0.48:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.58:44702 10.132.0.58:443 10.132.0.13:54962 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T14:56:40.462833Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:56:53.373754Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:57:10.988911Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:57:10.988949Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:57:10.988997Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T14:57:10.989009Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:57:10.989188Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T14:57:10.917Z] "POST /v1/messages HTTP/1.1" 404 - via_upstream - "-" 165 9 71 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "edd6d22b-7f67-4736-84dd-b1be12d1d23d" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.58:45442 10.132.0.58:443 10.132.0.13:37420 - v1-model-routing 2026-11-03T14:57:26.363199Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:57:26.363239Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:57:26.363252Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T14:57:26.363258Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:57:26.363418Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T14:57:26.302Z] "POST /v1/messages HTTP/1.1" 404 - via_upstream - "-" 165 9 60 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "21ceb616-c2f7-4277-8d2d-dfde222d56a0" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.58:45442 10.132.0.58:443 10.132.0.13:35684 - v1-model-routing 2026-11-03T14:57:34.207086Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T14:57:38.290Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 140 405 75 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "93ee5744-ef52-42e5-a1ee-fafeeddfc986" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.132.0.56:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.132.0.58:35492 10.132.0.58:443 10.132.0.13:54082 - v1-chat-completions-path 2026-11-03T14:57:49.835033Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:57:51.532226Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:57:57.541282Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:58:01.037755Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-11-03T14:58:00.962Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 75 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "19b18f33-5616-4981-b3fe-6d95a39c0d87" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.58:58046 10.132.0.58:443 10.132.0.13:58606 - v1-catch-all-path 2026-11-03T14:58:13.010908Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:58:14.706366Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:58:29.446270Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:58:40.300076Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:58:42.137694Z info ads ADS: new connection for node:6 2026-11-03T14:58:42.137939Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T14:58:42.138036Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T14:58:44.892715Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:59:24.417518Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:59:43.236154Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:59:43.708511Z info ads ADS: "" 6 terminated 2026-11-03T14:59:45.248376Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:59:48.283967Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:59:52.319041Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:02:07.046189Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:02:24.013872Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:02:39.943341Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:02:51.626637Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:02:55.309470Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:03:03.776948Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:03:06.278323Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T15:03:09.684Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 57 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "d352b658-113a-48a8-9634-eb5dfb64657a" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.58:443 10.132.0.13:59874 - v1-completions-path 2026-11-03T15:03:21.546542Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T15:03:37.496Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 74 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "368112bc-3432-4045-ad71-06edfb8dfd70" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.58:52308 10.132.0.58:443 10.132.0.13:49034 - v1-completions-path 2026-11-03T15:04:06.704849Z info ads ADS: new connection for node:7 2026-11-03T15:04:06.727379Z info cache read certificate from file resource=file-root:system 2026-11-03T15:04:07.206023Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:04:08.906075Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:04:25.208384Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T15:04:25.208415Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T15:04:25.208426Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T15:04:25.208430Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T15:04:25.208588Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T15:04:22.663Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1708 2545 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "9a68b244-fed5-4288-abf1-95815bad3a00" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.58:52324 10.132.0.58:443 10.132.0.13:42012 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T15:04:31.799529Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T15:04:31.799566Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T15:04:31.799578Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T15:04:31.799583Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T15:04:31.799734Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T15:04:31.678Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1711 121 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "60b2cc7b-c669-4661-8c84-5160a35f2b63" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.58:57532 10.132.0.58:443 10.132.0.13:49234 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T15:04:42.682075Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T15:04:42.682105Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T15:04:42.682114Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T15:04:42.682118Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T15:04:42.682265Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T15:04:42.580Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1710 101 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "98283a6b-62c0-49cb-9de1-72513591fad8" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.58:52324 10.132.0.58:443 10.132.0.13:57776 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T15:04:47.032824Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T15:04:47.032860Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T15:04:47.032871Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T15:04:47.032876Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T15:04:47.033135Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T15:04:46.934Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1704 98 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "7acf5cab-1e70-9033-ab90-e4965841c379" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.58:52324 10.132.0.58:443 10.132.0.13:51290 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T15:05:45.814631Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:05:45.969640Z info ads ADS: "" 7 terminated with stream closed 2026-11-03T15:05:46.255701Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T15:05:46.255748Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T15:05:46.255770Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T15:05:46.255777Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T15:05:46.479101Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T15:05:46.179Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 74 316 76 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "b7f7a51e-0a2e-4379-a0a2-5e70b43b1548" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.132.0.48:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.58:49964 10.132.0.58:443 10.132.0.13:44534 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T15:05:49.356471Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:05:51.626233Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:06:07.343875Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:06:09.026561Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:06:23.886232Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:06:25.662429Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:06:28.526369Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:06:37.306600Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T15:06:44.642Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 298 72 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "075b25a3-244e-47ac-ad5f-7007ff1b7bd2" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.58:36608 10.132.0.58:443 10.132.0.13:45360 - v1-completions-path [2026-11-03T15:06:44.897Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 63 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "b5cd0383-7fd3-461c-a066-ff128a97095a" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.58:36624 10.132.0.58:443 10.132.0.13:45366 - v1-completions-path 2026-11-03T15:07:01.977913Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:07:12.271082Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:07:14.597825Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T15:07:20.451Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 58 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "f87861c3-bd66-452f-b616-f94170851629" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.58:36608 10.132.0.58:443 10.132.0.13:34010 - v1-completions-path [2026-11-03T15:07:21.185Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 84 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "55ac50e5-5b98-4da8-8374-7f9069f78935" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.58:36624 10.132.0.58:443 10.132.0.13:34020 - v1-completions-path [2026-11-03T15:07:21.445Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 61 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "61933b5b-77f7-4fec-bd2f-336008e6b089" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.58:36608 10.132.0.58:443 10.132.0.13:34032 - v1-completions-path 2026-11-03T15:07:33.287974Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:07:35.074877Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:07:40.793207Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:07:51.773511Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:08:05.537717Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:08:07.320663Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:08:12.883973Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:08:30.731411Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:08:33.502203Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:08:48.062542Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:08:49.696612Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:08:51.454728Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T15:08:51.454765Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T15:08:51.454780Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T15:08:51.454785Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T15:08:51.147Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 35 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "a33711a9-3708-4563-8de4-84d0443010f6" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.58:443 10.132.0.13:42848 - v1-completions-path [2026-11-03T15:08:51.395Z] "DELETE /v1/api-keys/00c2be01-e982-4e22-b673-4f1da094640c HTTP/1.1" 200 - via_upstream - "-" 0 375 59 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "5a5c8cd9-baec-44d6-af55-89b7eec96e42" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.132.0.48:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.58:40160 10.132.0.58:443 10.132.0.13:42870 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T15:08:55.477290Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:09:10.614889Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:09:20.953825Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:09:37.033423Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:09:39.501288Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:11:24.451320Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:11:26.813013Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:12:48.295019Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:13:03.510204Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:13:05.287408Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:13:20.399481Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:13:30.220826Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:13:34.433911Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T15:13:37.716Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 63 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "22868741-a8de-4b32-87ad-fa78fd4e48f2" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.58:36624 10.132.0.58:443 10.132.0.13:34644 - v1-completions-path 2026-11-03T15:13:49.616053Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:14:00.092406Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:14:03.266804Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T15:14:07.818Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 60 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "bc538d0e-e1c6-44b7-8de4-9df829e6ba3d" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.58:36624 10.132.0.58:443 10.132.0.13:43182 - v1-completions-path 2026-11-03T15:14:20.078286Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:14:22.125465Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:14:25.226334Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T15:14:29.741Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 60 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "61997d82-c1d0-4d47-aa7b-449111123b4c" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.58:36624 10.132.0.58:443 10.132.0.13:50978 - v1-completions-path 2026-11-03T15:14:46.436807Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:14:56.595993Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:14:59.006314Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:15:02.215096Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:15:07.401947Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T15:15:11.103Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 60 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "cfec5661-f163-4401-8250-171fc993fc3d" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.58:36624 10.132.0.58:443 10.132.0.13:47538 - v1-completions-path 2026-11-03T15:15:23.663712Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:15:33.935833Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:15:35.605249Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:15:42.811947Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:15:45.522268Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:15:49.057143Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:16:06.172427Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:16:26.198414Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:16:36.336164Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:16:46.537308Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:16:56.819498Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:17:07.036378Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:17:17.252539Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:17:27.554754Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:17:37.723812Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:17:47.984648Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T15:17:58.171581Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T15:18:05.462Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 281 86 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "b4473556-42f8-4efc-98d8-445c637604d8" "maas.apps.d1482ea4acbc5dab920b.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.58:52308 10.132.0.58:443 10.132.0.13:39600 - v1-completions-path 2026-11-03T15:18:22.127563Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=4