2026-11-03T23:35:41.185082Z info FLAG: --concurrency="0" 2026-11-03T23:35:41.185106Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T23:35:41.185112Z info FLAG: --help="false" 2026-11-03T23:35:41.185116Z info FLAG: --log_as_json="false" 2026-11-03T23:35:41.185119Z info FLAG: --log_caller="" 2026-11-03T23:35:41.185123Z info FLAG: --log_output_level="default:info" 2026-11-03T23:35:41.185126Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T23:35:41.185134Z info FLAG: --log_target="[stdout]" 2026-11-03T23:35:41.185137Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T23:35:41.185141Z info FLAG: --outlierLogPath="" 2026-11-03T23:35:41.185144Z info FLAG: --profiling="true" 2026-11-03T23:35:41.185147Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T23:35:41.185150Z info FLAG: --proxyLogLevel="warning" 2026-11-03T23:35:41.185153Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T23:35:41.185157Z info FLAG: --stsPort="0" 2026-11-03T23:35:41.185160Z info FLAG: --templateFile="" 2026-11-03T23:35:41.185163Z info FLAG: --tokenManagerPlugin="" 2026-11-03T23:35:41.185167Z info FLAG: --vklog="0" 2026-11-03T23:35:41.185171Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T23:35:41.185376Z info Proxy role ips=[10.132.0.34] type=router id=maas-default-gateway-openshift-default-778b7b5c47-kchxb.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T23:35:41.185453Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T23:35:41.187327Z info cpu limit detected as 2, setting concurrency 2026-11-03T23:35:41.187610Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T23:35:41.187622Z info JWT policy is third-party-jwt 2026-11-03T23:35:41.187629Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T23:35:41.189599Z info Starting default Istio SDS Server 2026-11-03T23:35:41.189619Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T23:35:41.189583Z info Opening status port 15020 2026-11-03T23:35:41.189643Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T23:35:41.190357Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T23:35:41.192143Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T23:35:41.192932Z info Starting proxy agent 2026-11-03T23:35:41.192962Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T23:35:41.208773Z info sds Starting SDS grpc server 2026-11-03T23:35:41.208801Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T23:35:41.257626Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T23:35:41.345208Z info ads ADS: new connection for node:1 2026-11-03T23:35:41.345574Z info ads ADS: new connection for node:2 2026-11-03T23:35:41.362012Z info cache generated new workload certificate resourceName=default latency=171.441101ms ttl=23h59m59.637993185s 2026-11-03T23:35:41.362043Z info cache Root cert has changed, start rotating root cert 2026-11-03T23:35:41.362085Z info cache returned workload trust anchor from cache ttl=23h59m59.63791627s 2026-11-03T23:35:41.362130Z info cache returned workload certificate from cache ttl=23h59m59.637873545s 2026-11-03T23:35:41.362277Z info cache returned workload trust anchor from cache ttl=23h59m59.637724848s 2026-11-03T23:35:41.363124Z info cache returned workload trust anchor from cache ttl=23h59m59.636877051s 2026-11-03T23:35:41.406540Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T23:35:48.009529Z info Readiness succeeded in 6.828343775s 2026-11-03T23:35:48.009839Z info Envoy proxy is ready 2026-11-03T23:36:55.858751Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T23:36:56.559760Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T23:37:36.073059Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T23:37:38.890422Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T23:37:39.750066Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T23:37:41.562573Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T23:37:44.565555Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T23:37:47.346510Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T23:38:41.445Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 65 - "34.228.250.194,10.132.0.9" "curl/7.76.1" "1b6bbba9-c78b-44a6-8170-5440420b642f" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.34:443 10.132.0.9:59010 - v1-model-routing 2026-11-03T23:41:57.968399Z info ads ADS: new connection for node:3 2026-11-03T23:41:57.981127Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-03T23:41:57.981243Z info cache read certificate from file resource=file-root:system 2026-11-03T23:41:58.426255Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T23:42:03.427107Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T23:42:13.202620Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T23:42:16.601924Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:42:16.601957Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:42:16.601967Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T23:42:16.601972Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:42:16.602154Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 2026-11-03T23:42:16.764919Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:42:16.764960Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:42:16.764975Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T23:42:16.764987Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:42:16.765241Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T23:42:16.446Z] "POST /anything/e2e-rt-eq-0cb8b9-client HTTP/1.1" 200 - via_upstream - "-" 124 1581 155 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "51e55b27-50e2-4d1d-8ea5-6f17254c7753" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.34:33238 10.132.0.34:443 10.132.0.9:57286 - llm.external-model-e2e-rt-eq-0cb8b9-2304eb31a1d7.2 [2026-11-03T23:42:16.658Z] "POST /anything/e2e-rt-eq-0cb8b9-client HTTP/1.1" 200 - via_upstream - "-" 124 1581 106 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "713ee71f-7d3d-45c3-a703-d5c49256ef4e" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.34:33238 10.132.0.34:443 10.132.0.9:57296 - llm.external-model-e2e-rt-eq-0cb8b9-2304eb31a1d7.0 2026-11-03T23:42:20.246573Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T23:42:21.307164Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:42:21.307208Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:42:21.307236Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T23:42:21.307244Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:42:21.307513Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T23:42:21.037Z] "POST /anything/e2e-rt-eq-0cb8b9-client HTTP/1.1" 200 - via_upstream - "-" 124 1580 269 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "94af4a41-e15d-401a-bf24-7610cb4c5c33" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.34:33248 10.132.0.34:443 10.132.0.9:57300 - llm.external-model-e2e-rt-eq-0cb8b9-2304eb31a1d7.2 2026-11-03T23:42:24.623419Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:42:24.623463Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:42:24.623479Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T23:42:24.623485Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:42:24.623759Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 2026-11-03T23:42:25.361597Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:42:25.361729Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:42:25.361807Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T23:42:25.361837Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:42:25.362174Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T23:42:24.422Z] "POST /anything/e2e-rt-eq-0cb8b9-client HTTP/1.1" 200 - via_upstream - "-" 124 1576 201 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "08900efd-c2af-48c8-8a9a-98e40d435ff4" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.34:59412 10.132.0.34:443 10.132.0.9:35632 - llm.external-model-e2e-rt-eq-0cb8b9-2304eb31a1d7.2 [2026-11-03T23:42:25.181Z] "POST /anything/e2e-rt-eq-0cb8b9-client HTTP/1.1" 200 - via_upstream - "-" 124 1582 181 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "4e3bfc7d-d860-40bb-a28f-9b8cdf1e0b17" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.34:59416 10.132.0.34:443 10.132.0.9:35634 - llm.external-model-e2e-rt-eq-0cb8b9-2304eb31a1d7.2 2026-11-03T23:42:31.726563Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T23:42:36.112966Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-11-03T23:42:36.191107Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:42:36.193812Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:42:36.193917Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T23:42:36.193966Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:42:36.194307Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 2026-11-03T23:42:36.572498Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:42:36.572529Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:42:36.572539Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T23:42:36.572544Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:42:36.572694Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T23:42:36.040Z] "POST /anything/e2e-rt-sw-59f997-client HTTP/1.1" 200 - via_upstream - "-" 124 1582 154 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "92c04040-63ec-4ddf-a9b6-38934fe9b350" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.34:53416 10.132.0.34:443 10.132.0.9:44318 - llm.external-model-e2e-rt-sw-59f997-fb96a5702750.2 [2026-11-03T23:42:36.474Z] "POST /anything/e2e-rt-sw-59f997-client HTTP/1.1" 200 - via_upstream - "-" 124 1576 98 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "ae6280b3-92de-4113-8ccf-e43794412106" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.34:59416 10.132.0.34:443 10.132.0.9:44322 - llm.external-model-e2e-rt-sw-59f997-fb96a5702750.0 2026-11-03T23:42:44.929605Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T23:43:11.219837Z info ads ADS: "" 3 terminated 2026-11-03T23:43:11.359337Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T23:43:30.191109Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T23:44:59.747564Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T23:45:25.388422Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T23:46:52.290Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 83 0 43 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "78e9bd33-387e-4ea5-81d1-8a15e71c11c9" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.132.0.34:443 10.132.0.9:56478 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T23:46:54.315862Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:46:54.315898Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:46:54.315913Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T23:46:54.315917Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:46:54.456662Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:46:54.456702Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:46:54.456723Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:46:54.456731Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T23:46:55.066592Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:46:55.066628Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:46:55.066642Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T23:46:55.066647Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:46:55.128044Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-11-03T23:46:54.250Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 65 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "92b89391-150a-4214-b673-44a1fcbeb720" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:56428 10.132.0.34:443 10.132.0.9:56492 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T23:46:54.369Z] "DELETE /v1/api-keys/bd66d9ae-b096-43e1-b3ec-2f9b8e83f9e5 HTTP/1.1" 200 - via_upstream - "-" 0 321 87 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "b5de15f8-7a00-4342-ab0e-8f343edcb4eb" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:56428 10.132.0.34:443 10.132.0.9:56504 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T23:46:55.008Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 58 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "8f754207-7da5-44b7-a68b-fe5e2a8a154b" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:56428 10.132.0.34:443 10.132.0.9:56518 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T23:46:55.174792Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:46:55.174825Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:46:55.174839Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:46:55.174844Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T23:46:55.122Z] "DELETE /v1/api-keys/038cdd4a-21e8-474e-b596-06270016c271 HTTP/1.1" 200 - via_upstream - "-" 0 321 52 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "b9cf56c5-9848-4b0e-9528-93604303c56c" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:56440 10.132.0.34:443 10.132.0.9:56524 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T23:46:56.761833Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T23:47:05.764414Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:47:05.764455Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:47:05.764472Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:47:05.764478Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T23:47:05.689Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 316 75 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "c831cb04-3f15-46bb-84a3-a992e7c89f2e" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:56428 10.132.0.34:443 10.132.0.9:49100 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T23:47:24.999Z] "POST /maas-api/internal/v1/api-keys/validate HTTP/1.1" 404 NR route_not_found - "-" 31 0 16 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "33d32891-8f0a-4334-8d87-bb5c304fffb9" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "-" - - 10.132.0.34:443 10.132.0.9:45564 - - 2026-11-03T23:48:10.388282Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:01:54.066566Z info ads ADS: new connection for node:4 2026-11-04T00:01:54.072218Z info cache read certificate from file resource=file-root:system 2026-11-04T00:04:07.320319Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-04T00:10:59.682755Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:11:29.720587Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:12:02.372968Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:12:24.142587Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:12:51.351766Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:13:11.640490Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:13:49.871480Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:14:15.736044Z info ads ADS: new connection for node:5 2026-11-04T00:14:15.736431Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-04T00:14:15.736531Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-04T00:14:44.790815Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:15:05.784484Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:15:24.608558Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:17:50.579877Z info ads ADS: "" 5 terminated 2026-11-04T00:19:26.605419Z info ads ADS: "" 4 terminated with stream closed 2026-11-04T00:21:33.162608Z info cache event for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt : REMOVE, pushing to proxy 2026-11-04T00:24:45.290678Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:27:57.027539Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:28:31.195960Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:29:53.079668Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:35:18.924998Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=3 2026-11-04T00:39:12.449648Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:39:35.161796Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:40:08.889078Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-04T00:40:12.476Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 5518 5791 81 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "43d899c8-3648-4557-842c-960726bb384e" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.133.0.43:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:48490 10.132.0.34:443 10.132.0.9:33610 - v1-chat-completions-path 2026-11-04T00:40:24.675753Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:40:26.396071Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:40:33.018352Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:40:35.707263Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T00:40:35.707313Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T00:40:35.707332Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T00:40:35.707339Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-04T00:40:35.578Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 318 129 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "6cfa8276-e22a-482e-aade-4d855983256c" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:54268 10.132.0.34:443 10.132.0.9:36750 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T00:40:36.503Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 429 - - "-" 61 18 59 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "f99b3865-b466-4f8d-901a-65bcb94e4d00" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.34:443 10.132.0.9:36764 - v1-catch-all-path 2026-11-04T00:40:48.177608Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:40:49.880128Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:40:52.828927Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-11-04T00:40:52.752Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 76 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "7fcce42c-9850-4def-91cc-2d341aa6e3a5" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.26:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:44406 10.132.0.34:443 10.132.0.9:53234 - v1-catch-all-path 2026-11-04T00:40:59.361518Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=4 2026-11-04T00:41:04.394625Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:41:15.447900Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:41:17.698048Z info ads ADS: new connection for node:6 2026-11-04T00:41:17.698201Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-04T00:41:17.698230Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-04T00:41:19.840614Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:41:59.497121Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-04T00:42:02.621Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 331 102 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "7f58f96b-7b3d-4511-a837-ae8d3abe00a3" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.133.0.202:8000" outbound|54321||e2e-prio-pool-4fbc73a8-inference-pool-ip-89c5d4d5.llm.svc.cluster.local 10.132.0.34:35058 10.132.0.34:443 10.132.0.9:52006 - v1-model-routing 2026-11-04T00:42:28.649794Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:42:38.761369Z info ads ADS: "" 6 terminated 2026-11-04T00:42:40.721511Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:42:43.152345Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:42:54.347379Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:45:16.574269Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T00:45:16.574307Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T00:45:16.574321Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T00:45:16.574326Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-04T00:45:16.517Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 57 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "1e46cbac-0c80-4334-be66-00c776d0cfc6" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:47124 10.132.0.34:443 10.132.0.9:55686 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T00:45:18.242690Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T00:45:18.242723Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T00:45:18.242734Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T00:45:18.242764Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-04T00:45:18.189Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 53 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "2a54b4cd-8851-4052-9146-62a66ff4212b" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:47124 10.132.0.34:443 10.132.0.9:55696 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T00:45:23.158097Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:45:39.534769Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:45:54.654425Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:46:06.384383Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:46:10.131216Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:46:17.572936Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T00:46:17.572971Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T00:46:17.572983Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T00:46:17.572988Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T00:46:17.508Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 84 327 64 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "a49a5164-8afc-4bc3-b758-d6d31c6dfd43" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:55768 10.132.0.34:443 10.132.0.9:38946 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T00:46:29.178804Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:46:31.574964Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-04T00:46:35.110Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 61 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "912a40c5-9395-4a63-a9ea-1a8993df8ac7" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.34:443 10.132.0.9:49426 - v1-completions-path 2026-11-04T00:46:46.862239Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-04T00:47:12.937Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 290 72 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "8ebfd7ae-1de0-4d4f-8b12-81e88e2d3f37" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:40520 10.132.0.34:443 10.132.0.9:52078 - v1-completions-path 2026-11-04T00:48:02.093271Z info ads ADS: new connection for node:7 2026-11-04T00:48:02.097506Z info cache read certificate from file resource=file-root:system 2026-11-04T00:48:02.785214Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:48:04.805613Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:48:13.779169Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"1168","StreamId":"5863676710456886959"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=25 [2026-11-04T00:48:13.714Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 64 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "fe7a8f49-4e02-4bfc-9748-bcd4be8d696b" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.132.0.34:443 10.132.0.9:56616 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-11-04T00:48:28.288265Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T00:48:28.288298Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T00:48:28.288310Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T00:48:28.288315Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T00:48:28.288466Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-04T00:48:28.157Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1535 130 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "a7a10645-7785-4573-acd2-d0e149d9c75e" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.34:59882 10.132.0.34:443 10.132.0.9:35552 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-04T00:49:07.443981Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T00:49:07.444017Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T00:49:07.444032Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T00:49:07.444037Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T00:49:07.444219Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-04T00:49:07.347Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1530 96 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "d5ccebd9-ed22-4dd2-951b-b792ba25585b" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.34:59882 10.132.0.34:443 10.132.0.9:38278 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-04T00:49:40.491167Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:49:40.628042Z info ads ADS: "" 7 terminated with stream closed 2026-11-04T00:49:40.910773Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T00:49:40.910806Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T00:49:40.910820Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T00:49:40.910825Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T00:49:41.024458Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-04T00:49:41.024486Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-04T00:49:41.024491Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-04T00:49:41.024495Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 2026-11-04T00:49:41.045548Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-04T00:49:40.847Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 74 316 63 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "523dda72-8eb1-4be9-9f7e-1f74b89905fc" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:37994 10.132.0.34:443 10.132.0.9:41918 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T00:49:40.975Z] "POST /llm/premium-simulated-simulated-premium/v1/completions HTTP/1.1" 403 - - "-" 74 77 49 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "e552d284-a0b4-499f-bc60-55c77f7ced24" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "-" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.34:443 10.132.0.9:41920 - v1-completions-path 2026-11-04T00:49:43.865618Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:49:45.751071Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-04T00:49:49.413Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 73 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "935c179e-357e-40c8-9296-546bf88bf3c0" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:32876 10.132.0.34:443 10.132.0.9:57726 - v1-completions-path 2026-11-04T00:50:01.230134Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:50:03.077301Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:50:18.118475Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-04T00:50:17.474Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 69 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "ec105a1a-9675-46aa-97ec-07ddf79cce20" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:40520 10.132.0.34:443 10.132.0.9:53832 - v1-completions-path 2026-11-04T00:50:19.863118Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:50:22.691659Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:50:31.089466Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-04T00:50:37.815Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 63 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "7ff10f71-6992-4649-aa73-eed98fef3a86" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:32876 10.132.0.34:443 10.132.0.9:53298 - v1-completions-path [2026-11-04T00:50:38.500Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 57 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "e239fdc1-c017-432d-9f6e-07a6a94aefea" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:32876 10.132.0.34:443 10.132.0.9:53314 - v1-completions-path [2026-11-04T00:50:39.445Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 55 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "685c5fda-3cb4-49c9-ac6e-8b4eb18a20ab" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:32876 10.132.0.34:443 10.132.0.9:53332 - v1-completions-path [2026-11-04T00:50:40.186Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 62 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "aea134f9-e469-420a-a97c-d0fba1d7b6eb" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:46278 10.132.0.34:443 10.132.0.9:53338 - v1-completions-path [2026-11-04T00:50:40.808Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 300 60 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "d8f0c954-3c83-4718-94aa-8f88700d50c0" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:46278 10.132.0.34:443 10.132.0.9:53346 - v1-completions-path 2026-11-04T00:50:55.704636Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:51:06.007763Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:51:07.785379Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:51:26.760443Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:51:28.495063Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:51:34.156768Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:51:45.119517Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:51:58.987265Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:52:00.734205Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:52:06.339858Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:52:24.324999Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:52:35.084054Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:52:49.426969Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:52:51.186070Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:52:52.382193Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T00:52:52.382231Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T00:52:52.382249Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T00:52:52.382255Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-04T00:52:52.321Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 89 331 60 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "6c0f28b4-a836-4b78-b78e-f0e8b84f631f" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:32936 10.132.0.34:443 10.132.0.9:44214 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T00:52:52.555Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 32 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "762b0026-9647-4a46-b802-9163d10eed85" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.34:443 10.132.0.9:44230 - v1-completions-path 2026-11-04T00:52:57.058090Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-04T00:53:10.096Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 297 62 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "22444e56-d46f-47e5-ab46-0d922254a0d4" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:46278 10.132.0.34:443 10.132.0.9:40114 - v1-completions-path 2026-11-04T00:53:21.986802Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:53:32.266425Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:54:07.121354Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T00:54:07.121388Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T00:54:07.121401Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T00:54:07.121406Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-04T00:54:07.060Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 61 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "5add9d86-020d-466c-a992-396d70b07782" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:39888 10.132.0.34:443 10.132.0.9:41746 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T00:54:17.619Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 45 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "bfe650ab-9c05-48bc-99d2-023f3ce7f704" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.34:443 10.132.0.9:40536 - v1-completions-path 2026-11-04T00:54:19.280874Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:54:22.193267Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:57:08.809825Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:57:11.623908Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:58:31.873050Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:58:44.418317Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T00:58:44.418350Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T00:58:44.418363Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T00:58:44.418368Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T00:58:44.362Z] "DELETE /v1/api-keys/e2e-trlp-test-key HTTP/1.1" 404 - via_upstream - "-" 0 29 56 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "2b45a12d-5643-4c79-9ba6-43a35369be48" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:40238 10.132.0.34:443 10.132.0.9:60016 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T00:58:55.955393Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:58:57.714101Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:59:03.637229Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:59:22.454358Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:59:26.466249Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:59:30.907014Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T00:59:30.907050Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T00:59:30.907061Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T00:59:30.907066Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T00:59:31.026895Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T00:59:31.026928Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T00:59:31.026941Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T00:59:31.026945Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-04T00:59:30.728Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 63 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "6c15cad0-2501-4d9a-8630-3b74ef859681" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:46278 10.132.0.34:443 10.132.0.9:38188 - v1-completions-path [2026-11-04T00:59:30.844Z] "DELETE /v1/api-keys/e5ac551c-a0cc-4930-aa8c-32286d351983 HTTP/1.1" 200 - via_upstream - "-" 0 369 62 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "4bbdb2af-cf97-40ee-82b0-759c17f56a59" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.133.0.245:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:42626 10.132.0.34:443 10.132.0.9:38194 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T00:59:30.964Z] "DELETE /v1/api-keys/c3e4696b-e68d-473a-8409-8ae71ef37696 HTTP/1.1" 200 - via_upstream - "-" 0 369 62 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "bbc08a2d-1ee0-48e8-a537-8b8139f1fc19" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.243:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:45818 10.132.0.34:443 10.132.0.9:38198 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T00:59:42.018914Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T00:59:52.405801Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:00:04.889573Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-04T01:00:09.304Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 59 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "92ca133d-7e3f-4421-bf6c-74608cd0edb4" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:46278 10.132.0.34:443 10.132.0.9:35314 - v1-completions-path [2026-11-04T01:00:09.975Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 56 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "3ba9fea7-936e-4f22-aa8a-c71b83005304" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:46278 10.132.0.34:443 10.132.0.9:35330 - v1-completions-path 2026-11-04T01:00:22.430379Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:00:34.498614Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:00:38.481972Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-04T01:00:44.504Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 68 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "3ed27652-df68-4668-b5de-b3fd1a331898" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:46278 10.132.0.34:443 10.132.0.9:44844 - v1-completions-path [2026-11-04T01:00:46.271Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 63 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "0b6ebd4e-f224-4032-800a-237ef55de9db" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:32876 10.132.0.34:443 10.132.0.9:44860 - v1-completions-path [2026-11-04T01:00:46.938Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 64 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "ed82bf7d-7713-43a4-927a-1cda48f0b792" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:46278 10.132.0.34:443 10.132.0.9:44870 - v1-completions-path [2026-11-04T01:00:47.612Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 65 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "00652412-5b53-4d1a-a568-5e98564cd55a" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:32876 10.132.0.34:443 10.132.0.9:44880 - v1-completions-path [2026-11-04T01:00:48.054Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 64 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "8bf46b04-8f46-4a5e-a323-fb909df92802" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.34:443 10.132.0.9:44892 - v1-completions-path 2026-11-04T01:01:00.059271Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:01:10.207243Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:01:12.462756Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:01:15.592565Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-04T01:01:20.215Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 64 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "36e2cff4-8721-43d6-93ea-4f0e06083a9e" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:46278 10.132.0.34:443 10.132.0.9:35148 - v1-completions-path 2026-11-04T01:01:21.805191Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:01:38.265400Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:01:48.403605Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:01:50.097507Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:02:10.488727Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:02:13.478367Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:02:16.760646Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:02:32.784108Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:02:52.731043Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:03:02.918074Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:03:13.192364Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:03:23.430076Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:03:33.636424Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:03:43.987606Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:03:54.103548Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:04:04.362259Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:04:14.568354Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:04:24.780637Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-04T01:04:38.200Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 289 61 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "ac75945b-944c-4aa6-84f0-b91a0252eb16" "maas.apps.b4341b0d6c25eae239a7.prod.konflux-ocp-ci.dev" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:40520 10.132.0.34:443 10.132.0.9:32784 - v1-completions-path