2026-11-03T13:17:54.753493Z info FLAG: --concurrency="0" 2026-11-03T13:17:54.753514Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T13:17:54.753518Z info FLAG: --help="false" 2026-11-03T13:17:54.753521Z info FLAG: --log_as_json="false" 2026-11-03T13:17:54.753523Z info FLAG: --log_caller="" 2026-11-03T13:17:54.753525Z info FLAG: --log_output_level="default:info" 2026-11-03T13:17:54.753527Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T13:17:54.753531Z info FLAG: --log_target="[stdout]" 2026-11-03T13:17:54.753533Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T13:17:54.753535Z info FLAG: --outlierLogPath="" 2026-11-03T13:17:54.753537Z info FLAG: --profiling="true" 2026-11-03T13:17:54.753539Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T13:17:54.753541Z info FLAG: --proxyLogLevel="warning" 2026-11-03T13:17:54.753543Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T13:17:54.753545Z info FLAG: --stsPort="0" 2026-11-03T13:17:54.753547Z info FLAG: --templateFile="" 2026-11-03T13:17:54.753549Z info FLAG: --tokenManagerPlugin="" 2026-11-03T13:17:54.753551Z info FLAG: --vklog="0" 2026-11-03T13:17:54.753554Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T13:17:54.753695Z info Proxy role ips=[10.132.0.46] type=router id=maas-default-gateway-openshift-default-84587756dc-vrbqc.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T13:17:54.753742Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T13:17:54.754746Z info cpu limit detected as 2, setting concurrency 2026-11-03T13:17:54.754935Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T13:17:54.754945Z info JWT policy is third-party-jwt 2026-11-03T13:17:54.754948Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T13:17:54.756142Z info Starting default Istio SDS Server 2026-11-03T13:17:54.756166Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T13:17:54.756183Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T13:17:54.756617Z info Opening status port 15020 2026-11-03T13:17:54.756993Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T13:17:54.757430Z info sds Starting SDS grpc server 2026-11-03T13:17:54.757449Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T13:17:54.758586Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T13:17:54.760101Z info Starting proxy agent 2026-11-03T13:17:54.761246Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T13:17:54.827472Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T13:17:54.859029Z info cache generated new workload certificate resourceName=default latency=101.255136ms ttl=23h59m59.140974906s 2026-11-03T13:17:54.859066Z info cache Root cert has changed, start rotating root cert 2026-11-03T13:17:54.859105Z info cache returned workload trust anchor from cache ttl=23h59m59.140896272s 2026-11-03T13:17:54.953778Z info ads ADS: new connection for node:1 2026-11-03T13:17:54.953875Z info cache returned workload certificate from cache ttl=23h59m59.046142238s 2026-11-03T13:17:54.954636Z info ads ADS: new connection for node:2 2026-11-03T13:17:54.954703Z info cache returned workload trust anchor from cache ttl=23h59m59.045297729s 2026-11-03T13:17:55.040628Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:18:02.905450Z info Readiness succeeded in 8.154212413s 2026-11-03T13:18:02.905903Z info Envoy proxy is ready 2026-11-03T13:18:37.059029Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:18:40.747086Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:20:26.227394Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:20:26.227549Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:20:26.227604Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T13:20:26.227636Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T13:20:26.162Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 91 64 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "f5224ff9-3bd4-413d-8f14-0039d8d2656c" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.133.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.46:35126 10.132.0.46:443 10.132.0.13:35126 - odh-ai-gateway-infra.maas-api-route.4 2026-11-03T13:20:26.537570Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:20:26.537609Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:20:26.537626Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:20:26.537631Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T13:20:26.312Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 225 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "7257d8e3-ea13-4ae9-98d8-4d5a8e000b39" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.133.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.46:35128 10.132.0.46:443 10.132.0.13:35134 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T13:20:26.692Z] "POST /maas-api/v1/tokens HTTP/1.1" 401 - - "-" 20 88 32 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "57e95517-9262-4bcb-9e42-d579fbda8b5e" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.132.0.46:443 10.132.0.13:35146 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T13:21:43.883160Z info ads ADS: new connection for node:3 2026-11-03T13:21:43.891171Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-03T13:21:43.891320Z info cache read certificate from file resource=file-root:system 2026-11-03T13:22:51.578653Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:22:51.578699Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:22:51.578720Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:22:51.578727Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T13:22:51.510Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 72 315 67 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "c361372a-ef44-47eb-b344-5b2fa6bff226" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.133.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.46:35624 10.132.0.46:443 10.132.0.13:46460 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T13:28:56.489014Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:29:08.641835Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:29:28.091524Z info ads ADS: new connection for node:4 2026-11-03T13:29:28.091844Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T13:29:28.091939Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T13:29:56.662693Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:30:02.731454Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:30:02.731492Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:30:02.731503Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:30:02.731510Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T13:30:02.731759Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T13:30:02.562Z] "POST /anything/e2e-dft-f2efc0-f581b0 HTTP/1.1" 200 - via_upstream - "-" 109 1769 168 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "31a1f0eb-ec95-4c7c-89ee-fe0ae7160ae3" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.46:49568 10.132.0.46:443 10.132.0.13:47066 - llm.external-model-e2e-dft-f2efc0-e3527cf7ba61.0 2026-11-03T13:30:36.415212Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:31:04.461936Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:31:31.740515Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:31:36.726161Z info ads ADS: "" 4 terminated 2026-11-03T13:31:55.945634Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:32:00.968185Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:32:00.968227Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:32:00.968242Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T13:32:00.968248Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:32:00.968552Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T13:32:00.816Z] "POST /anything/e2e-rt-sw-2ece70-client HTTP/1.1" 200 - via_upstream - "-" 124 1752 152 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "176fd048-b352-4af5-8e1d-1d4360f75c25" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.46:47446 10.132.0.46:443 10.132.0.13:46518 - llm.external-model-e2e-rt-sw-2ece70-8a582f45b424.0 2026-11-03T13:32:12.426226Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:32:36.889835Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:33:02.938960Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:34:42.183374Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:34:47.622726Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:34:47.622760Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:34:47.622770Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:34:47.622775Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T13:34:47.622945Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 2026-11-03T13:34:48.089483Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:34:48.089628Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:34:48.089654Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T13:34:48.089662Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:34:48.089907Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T13:34:47.487Z] "POST /anything/e2e-rt-eq-86abdb-client HTTP/1.1" 200 - via_upstream - "-" 124 1762 135 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "6b1102cf-5438-486b-b430-bfdad88f1cd6" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.46:42128 10.132.0.46:443 10.132.0.13:38586 - llm.external-model-e2e-rt-eq-86abdb-c702f89b7e8b.0 [2026-11-03T13:34:47.959Z] "POST /anything/e2e-rt-eq-86abdb-client HTTP/1.1" 200 - via_upstream - "-" 124 1754 130 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "4581921c-3903-44a5-8c7b-cee679031e9d" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.46:42136 10.132.0.46:443 10.132.0.13:38590 - llm.external-model-e2e-rt-eq-86abdb-c702f89b7e8b.0 2026-11-03T13:34:50.023490Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:34:50.023520Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:34:50.023528Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T13:34:50.023533Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:34:50.023689Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T13:34:49.924Z] "POST /anything/e2e-rt-eq-86abdb-client HTTP/1.1" 200 - via_upstream - "-" 124 1760 99 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "ba8f050a-56d7-4baa-a541-fe74e9faf3af" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.46:42128 10.132.0.46:443 10.132.0.13:38604 - llm.external-model-e2e-rt-eq-86abdb-c702f89b7e8b.2 2026-11-03T13:35:15.170991Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:35:31.771493Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:35:53.602181Z info ads ADS: "" 3 terminated with stream closed 2026-11-03T13:36:34.705358Z info ads ADS: new connection for node:5 2026-11-03T13:36:34.711458Z info cache read certificate from file resource=file-root:system 2026-11-03T13:37:18.910667Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:38:08.260548Z info ads ADS: "" 5 terminated 2026-11-03T13:39:05.073410Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:42:23.411102Z info ads ADS: new connection for node:6 2026-11-03T13:42:23.423195Z info cache read certificate from file resource=file-root:system 2026-11-03T13:44:21.201867Z info ads ADS: "" 6 terminated 2026-11-03T13:46:09.467541Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:46:20.550376Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-03T13:46:33.952388Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:48:35.705973Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:48:50.918024Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:48:52.623108Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:49:02.609701Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:49:05.937682Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:49:09.445071Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:49:24.757704Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:49:44.647851Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:49:54.870900Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:50:05.185531Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:50:08.551381Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=3 2026-11-03T13:50:15.516546Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:50:25.641045Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:50:35.907608Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:50:46.097416Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:50:56.275299Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:51:06.454203Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:51:16.719599Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:51:54.119045Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:52:10.082651Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:52:11.737572Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:52:15.127793Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-03T13:52:15.210785Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-11-03T13:52:15.119Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 91 3616 90 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "a2c63251-1a95-4587-b8d5-50d1dedcfbc1" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.134.0.31:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.46:32850 10.132.0.46:443 10.132.0.13:59394 - v1-catch-all-model-routing 2026-11-03T13:52:17.537211Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:52:33.040157Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:52:34.656711Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:52:49.234071Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:52:59.936952Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:53:02.018178Z info ads ADS: new connection for node:7 2026-11-03T13:53:02.018499Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T13:53:02.922329Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:53:43.841856Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:54:02.786088Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:54:03.303713Z info ads ADS: "" 7 terminated 2026-11-03T13:54:04.599692Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:54:07.736035Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:54:11.668940Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:54:23.819510Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:54:40.495123Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:54:56.775969Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:55:08.409711Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:55:12.228721Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:55:20.336666Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:55:22.938103Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:55:25.918595Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-11-03T13:55:25.947361Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:55:25.947499Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:55:25.947549Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:55:25.947576Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T13:55:25.711Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 336 236 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "0b8c9799-dcbc-439d-9266-6669dd006d00" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.133.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.46:54242 10.132.0.46:443 10.132.0.13:49086 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T13:55:38.588707Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:59:40.791583Z info ads ADS: new connection for node:8 2026-11-03T13:59:40.798299Z info cache read certificate from file resource=file-root:system 2026-11-03T13:59:41.469231Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:59:42.939842Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:59:52.651974Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"691","StreamId":"7078950499733319679"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=24 [2026-11-03T13:59:52.590Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 61 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "a694229e-39e8-4bb0-b49f-74671a19c11e" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.132.0.46:443 10.132.0.13:39654 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-11-03T14:00:03.670631Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:00:03.670663Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:00:03.670675Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:00:03.670679Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T14:00:03.670862Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T14:00:03.541Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1706 129 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "67ca9365-7606-4848-8e42-bdb4c52c68bc" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.46:46584 10.132.0.46:443 10.132.0.13:46938 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T14:00:23.263887Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:00:23.263918Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:00:23.263927Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:00:23.263932Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T14:00:23.264110Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T14:00:23.167Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1706 96 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "1312ec80-04b3-44f0-93ca-21584d466fa4" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.46:46584 10.132.0.46:443 10.132.0.13:51752 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T14:00:45.010940Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:00:45.010971Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:00:45.010980Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T14:00:45.010984Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:00:45.011136Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T14:00:44.881Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1712 129 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "17323ab4-b54d-4137-82e3-6cdc221e9b47" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.46:44792 10.132.0.46:443 10.132.0.13:58756 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T14:00:47.209110Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:00:47.209159Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:00:47.209174Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:00:47.209179Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T14:00:47.209458Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T14:00:47.076Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1712 133 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "8da965af-8b86-4103-b550-0b42971cfec7" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.46:46584 10.132.0.46:443 10.132.0.13:58764 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T14:00:58.024948Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:00:58.024984Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:00:58.024996Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T14:00:58.025000Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:00:58.025187Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T14:00:57.920Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1712 104 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "981ed234-0892-4b9f-8de3-0f0943375b5c" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.46:46584 10.132.0.46:443 10.132.0.13:34326 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T14:01:13.242388Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:01:13.242419Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:01:13.242429Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T14:01:13.242433Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:01:13.242622Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T14:01:13.143Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1708 98 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "06819831-ff55-40b2-b350-7ee30a713f30" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.46:44792 10.132.0.46:443 10.132.0.13:47340 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T14:01:26.670247Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:01:26.796570Z info ads ADS: "" 8 terminated 2026-11-03T14:01:27.102379Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:01:27.102412Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:01:27.102428Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:01:27.102433Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T14:01:27.030Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 74 316 72 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "050df996-b0b0-4cfe-bd25-7139b786ea2e" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.133.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.46:39676 10.132.0.46:443 10.132.0.13:51730 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T14:01:28.349983Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:01:31.457142Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:01:43.418154Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:01:45.257682Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:01:48.659280Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:01:56.683143Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T14:02:02.912Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 72 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "6e54ea57-474f-4e9c-8e02-5e0ca8b9f0f1" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.132.0.41:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.46:50382 10.132.0.46:443 10.132.0.13:34720 - v1-completions-path [2026-11-03T14:02:03.722Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 57 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "b6b4410f-7a6e-4d40-823c-e581633363a0" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.132.0.41:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.46:50386 10.132.0.46:443 10.132.0.13:34730 - v1-completions-path [2026-11-03T14:02:05.017Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 58 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "ab8adfe0-0b86-4829-ac3a-4ca38101b083" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.132.0.41:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.46:50382 10.132.0.46:443 10.132.0.13:50752 - v1-completions-path [2026-11-03T14:02:06.451Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 60 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "4ff05072-6f0a-4a27-a5a8-4a04af068301" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.132.0.41:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.46:50386 10.132.0.46:443 10.132.0.13:50766 - v1-completions-path 2026-11-03T14:02:21.377765Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:02:31.656648Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:02:33.291874Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T14:02:39.280Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 60 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "2c2db9de-b317-4dad-b5be-7a0afd7d21f5" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.132.0.41:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.46:50382 10.132.0.46:443 10.132.0.13:33720 - v1-completions-path [2026-11-03T14:02:40.198Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 56 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "a93ef00f-ff0b-4e39-84c9-c5e55b6bd214" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.132.0.41:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.46:50382 10.132.0.46:443 10.132.0.13:33730 - v1-completions-path 2026-11-03T14:02:52.321614Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:02:54.015756Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T14:02:57.179Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 74 297 98 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "ef11ec0a-da0b-4239-83d2-e14b7dace49d" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.132.0.46:52726 10.132.0.46:443 10.132.0.13:39460 - v1-completions-path 2026-11-03T14:02:59.865519Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:03:10.788143Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:03:24.469972Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:03:26.090729Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:03:31.748363Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T14:03:38.228Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 75 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "39aa32fa-7344-452d-9fff-0120f1ecee8f" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.46:40746 10.132.0.46:443 10.132.0.13:38178 - v1-completions-path 2026-11-03T14:03:49.799562Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:03:52.833780Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T14:03:53.144Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 288 64 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "d5e35fb3-9b29-42e1-b768-21364cad83d9" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.46:40746 10.132.0.46:443 10.132.0.13:33698 - v1-completions-path 2026-11-03T14:04:05.235908Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:04:07.296573Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:04:11.824907Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T14:04:18.416Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 58 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "599bcd7a-9782-4700-a113-96e898060caa" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.132.0.41:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.46:50382 10.132.0.46:443 10.132.0.13:37066 - v1-completions-path 2026-11-03T14:04:30.151696Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:04:40.408347Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:04:56.909882Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:04:59.822130Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:06:42.193924Z info cache event for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt : REMOVE, pushing to proxy 2026-11-03T14:07:14.485850Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:07:17.474469Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:07:19.704945Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:07:34.813240Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:07:45.037151Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:08:05.013214Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:11:40.706351Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:11:45.093003Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T14:11:48.104Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 70 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "8f01fa7f-1bc8-4efd-b9b9-e8acad61906c" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.132.0.41:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.46:50382 10.132.0.46:443 10.132.0.13:57744 - v1-completions-path [2026-11-03T14:11:49.059Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 56 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "e629c20f-4b64-4340-bed8-2b708be7ed86" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.132.0.41:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.46:50382 10.132.0.46:443 10.132.0.13:57760 - v1-completions-path 2026-11-03T14:12:01.294024Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:12:11.444615Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:12:14.780202Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:12:33.364260Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:12:35.535688Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:12:38.835815Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T14:12:45.128Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 58 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "0428b896-ec5e-4213-8da6-e3f5f14e0e9e" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.132.0.41:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.46:50386 10.132.0.46:443 10.132.0.13:53454 - v1-completions-path [2026-11-03T14:12:45.351Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 62 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "8c8a0437-653c-4152-94be-4e4164811c02" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.132.0.41:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.46:50382 10.132.0.46:443 10.132.0.13:53468 - v1-completions-path [2026-11-03T14:12:46.747Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 59 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "be585632-ea9f-4d60-ad72-708dd3418ab5" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.132.0.41:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.46:50386 10.132.0.46:443 10.132.0.13:53474 - v1-completions-path 2026-11-03T14:12:59.629987Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:13:09.890534Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:13:12.067957Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:13:15.468853Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:13:23.558182Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:13:42.256296Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:13:52.395705Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:13:55.296521Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T14:13:55.296546Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T14:13:55.296549Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T14:13:55.296553Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-11-03T14:13:55.272Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 401 - - "-" 66 88 24 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "7260641b-3b24-4313-859f-de981aff2703" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.46:443 10.132.0.13:36436 - v1-completions-path 2026-11-03T14:13:57.911569Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T14:13:57.911604Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T14:13:57.911608Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T14:13:57.911613Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-11-03T14:13:57.357Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 288 83 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "b7ce7c14-613b-4b38-9708-d71b59b63374" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.46:53128 10.132.0.46:443 10.132.0.13:36458 - v1-completions-path [2026-11-03T14:13:57.508Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 79 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "a00294ef-3f09-4643-a275-27ec365fc519" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.46:40746 10.132.0.46:443 10.132.0.13:36460 - v1-completions-path [2026-11-03T14:13:57.872Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 401 - - "-" 66 88 39 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "a5dab368-f69a-4627-92d4-01a597d118f5" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.46:443 10.132.0.13:36466 - v1-completions-path