2026-11-03T13:22:18.316244Z info FLAG: --concurrency="0" 2026-11-03T13:22:18.316260Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T13:22:18.316265Z info FLAG: --help="false" 2026-11-03T13:22:18.316268Z info FLAG: --log_as_json="false" 2026-11-03T13:22:18.316271Z info FLAG: --log_caller="" 2026-11-03T13:22:18.316273Z info FLAG: --log_output_level="default:info" 2026-11-03T13:22:18.316277Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T13:22:18.316283Z info FLAG: --log_target="[stdout]" 2026-11-03T13:22:18.316286Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T13:22:18.316289Z info FLAG: --outlierLogPath="" 2026-11-03T13:22:18.316293Z info FLAG: --profiling="true" 2026-11-03T13:22:18.316297Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T13:22:18.316300Z info FLAG: --proxyLogLevel="warning" 2026-11-03T13:22:18.316303Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T13:22:18.316306Z info FLAG: --stsPort="0" 2026-11-03T13:22:18.316309Z info FLAG: --templateFile="" 2026-11-03T13:22:18.316311Z info FLAG: --tokenManagerPlugin="" 2026-11-03T13:22:18.316315Z info FLAG: --vklog="0" 2026-11-03T13:22:18.316319Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T13:22:18.316465Z info Proxy role ips=[10.132.0.229] type=router id=maas-default-gateway-openshift-default-7f5667f86c-f2bh6.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T13:22:18.316515Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T13:22:18.317619Z info cpu limit detected as 2, setting concurrency 2026-11-03T13:22:18.318142Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T13:22:18.318170Z info JWT policy is third-party-jwt 2026-11-03T13:22:18.318175Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T13:22:18.320680Z info Opening status port 15020 2026-11-03T13:22:18.320729Z info Starting default Istio SDS Server 2026-11-03T13:22:18.320754Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T13:22:18.320778Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T13:22:18.321399Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T13:22:18.322738Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T13:22:18.323530Z info Starting proxy agent 2026-11-03T13:22:18.323557Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T13:22:18.327572Z info sds Starting SDS grpc server 2026-11-03T13:22:18.327590Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T13:22:18.386416Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T13:22:18.436187Z info cache generated new workload certificate resourceName=default latency=114.022147ms ttl=23h59m59.563817465s 2026-11-03T13:22:18.436224Z info cache Root cert has changed, start rotating root cert 2026-11-03T13:22:18.436264Z info cache returned workload trust anchor from cache ttl=23h59m59.563736569s 2026-11-03T13:22:18.538223Z info ads ADS: new connection for node:1 2026-11-03T13:22:18.538300Z info ads ADS: new connection for node:2 2026-11-03T13:22:18.538302Z info cache returned workload certificate from cache ttl=23h59m59.461700275s 2026-11-03T13:22:18.538380Z info cache returned workload trust anchor from cache ttl=23h59m59.461621464s 2026-11-03T13:22:18.624305Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:22:26.614804Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:22:27.557904Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T13:22:27.827830Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T13:22:29.518135Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:22:30.259849Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T13:22:30.427292Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T13:22:30.937845Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T13:22:32.046405Z info Readiness succeeded in 13.733126952s 2026-11-03T13:22:32.046756Z info Envoy proxy is ready 2026-11-03T13:22:37.283296Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:22:42.313749Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-11-03T13:22:42.230Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 83 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "c6a2e838-19ab-41c6-b73a-8feef8e2a516" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "10.132.0.35:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.229:47446 10.132.0.229:443 10.132.0.20:50954 - v1-catch-all-path 2026-11-03T13:22:50.958502Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:22:59.548131Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:23:16.372835Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:23:28.090719Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:23:32.039974Z info ads ADS: new connection for node:3 2026-11-03T13:23:32.040174Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T13:23:32.040291Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T13:23:35.055978Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:24:19.199836Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T13:24:41.227Z] "POST /v1/chat/completions HTTP/1.1" 403 - - "-" 131 0 34 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "8f86b28e-9abc-49b2-a0fc-5c44542833da" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "-" outbound|54321||e2e-prio-pool-b2d3d219-inference-pool-ip-d3e9463c.llm.svc.cluster.local - 10.132.0.229:443 10.132.0.20:57612 - v1-model-routing 2026-11-03T13:24:43.359886Z info ads ADS: "" 3 terminated with stream closed 2026-11-03T13:24:43.719968Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:24:48.656378Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:25:05.082623Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:27:37.101371Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:28:00.839383Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:28:27.035315Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:28:42.392506Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:28:50.849637Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:28:57.452894Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:29:06.839380Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:29:12.421117Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:29:15.283763Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:29:15.283799Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:29:15.283813Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:29:15.283818Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T13:29:15.533832Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:29:15.533870Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:29:15.533886Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T13:29:15.533891Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T13:29:15.165Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 335 118 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "9f720813-64bb-4e10-b79f-f5b3dcd1e127" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "10.132.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.229:38734 10.132.0.229:443 10.132.0.20:48874 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T13:29:15.475Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 524 58 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "907ce296-3ff2-4b8e-bcc9-6d25433ef0b4" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "10.132.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.229:38748 10.132.0.229:443 10.132.0.20:48876 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T13:29:16.009Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 75 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "a36a32fc-7d74-4ba6-aab9-43b1cf4936fc" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "10.132.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.229:46312 10.132.0.229:443 10.132.0.20:48884 - v1-completions-path 2026-11-03T13:29:31.846474Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:30:19.950662Z info ads ADS: new connection for node:4 2026-11-03T13:30:19.955860Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-03T13:30:19.955898Z info cache read certificate from file resource=file-root:system 2026-11-03T13:30:23.828496Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:30:24.954115Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:31:03.353721Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-11-03T13:31:03.489532Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:31:03.489563Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:31:03.489573Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T13:31:03.489578Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:31:03.489741Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T13:31:03.291Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1714 198 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "4e91eb47-50e2-43d5-a9e8-1ba08443b2b3" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.229:60006 10.132.0.229:443 10.132.0.20:52218 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T13:31:48.872544Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:31:48.872575Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:31:48.872589Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:31:48.872595Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T13:31:48.872835Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T13:31:48.742Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1713 130 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "5af38de8-8d95-4d7a-9e89-46113617c72e" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.229:60006 10.132.0.229:443 10.132.0.20:52732 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T13:32:08.833372Z info ads ADS: "" 4 terminated with stream closed 2026-11-03T13:32:08.882170Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:32:11.488861Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:32:22.027196Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:32:40.219943Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:32:44.855297Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:32:47.738312Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:32:47.738346Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:32:47.738359Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:32:47.738364Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T13:32:47.625Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 335 112 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "8c809459-d326-42cf-b93b-0aabc097c6d1" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "10.132.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.229:57040 10.132.0.229:443 10.132.0.20:43794 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T13:32:49.752Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 73 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "4bee73e0-a8ea-4d52-997e-dde83d390bd7" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "10.132.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.229:40342 10.132.0.229:443 10.132.0.20:55360 - v1-completions-path 2026-11-03T13:33:08.922223Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:33:22.102126Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:33:52.366134Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:34:02.995136Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:34:09.453760Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T13:34:21.371Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 61 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "dfbbfd03-3a48-46bd-81d4-ba5d8b6513d6" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "10.132.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.229:40342 10.132.0.229:443 10.132.0.20:56590 - v1-completions-path [2026-11-03T13:34:21.620Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 55 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "40e44777-1f64-4458-a3cf-dc57a40b24ca" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "10.132.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.229:46312 10.132.0.229:443 10.132.0.20:56592 - v1-completions-path 2026-11-03T13:34:37.613299Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:34:42.334571Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:34:55.790960Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:35:00.729647Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-11-03T13:35:00.669Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 299 76 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "1c21f782-c5bb-4395-b05a-73cf418dc0b7" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.229:59118 10.132.0.229:443 10.132.0.20:34958 - v1-completions-path 2026-11-03T13:35:05.864362Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:35:15.307887Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:35:24.426186Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:35:33.834726Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:35:51.108030Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:35:57.019033Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:36:15.108503Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:36:20.591802Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T13:36:22.143Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 34 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "7e6e9153-3ab6-4844-b7b2-b3bac9b403e9" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.229:443 10.132.0.20:37910 - v1-completions-path 2026-11-03T13:36:22.767651Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:36:22.767688Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:36:22.767720Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T13:36:22.767725Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T13:36:22.660Z] "DELETE /v1/api-keys/826c22e9-4dfd-4ecb-8e16-d746bdb27271 HTTP/1.1" 200 - via_upstream - "-" 0 375 107 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "8d907bb0-7ce2-498c-af19-10990bf8579d" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "10.132.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.229:41750 10.132.0.229:443 10.132.0.20:37926 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T13:36:32.307106Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:36:58.447787Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:37:09.078416Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:37:33.211216Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:37:39.205200Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:40:38.021601Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:42:06.264086Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:42:23.390749Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:42:28.024600Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:42:31.056333Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:42:31.056369Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:42:31.056385Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:42:31.056389Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T13:42:28.862Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 65 308 2194 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "5430e43a-2076-4da7-ac43-eff6c72a76ec" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "10.132.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.229:59102 10.132.0.229:443 10.132.0.20:49800 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T13:42:40.608501Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:42:58.088480Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:43:04.314937Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T13:43:07.717Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 74 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "471ca02b-65ef-4114-a31c-74de7d67d801" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "10.132.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.229:40342 10.132.0.229:443 10.132.0.20:42766 - v1-completions-path [2026-11-03T13:43:07.980Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 54 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "dfc25acb-6f4a-457f-9675-3dfd854405f1" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "10.132.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.229:40342 10.132.0.229:443 10.132.0.20:42782 - v1-completions-path 2026-11-03T13:43:10.251495Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:43:10.251533Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:43:10.251548Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T13:43:10.251553Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T13:43:10.144Z] "DELETE /v1/api-keys/078afa02-9a1d-4752-be04-64b526b05209 HTTP/1.1" 200 - via_upstream - "-" 0 369 107 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "9cde4344-2858-45f2-9a80-ea911252dda8" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "10.134.0.168:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.229:51734 10.132.0.229:443 10.132.0.20:42792 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T13:43:24.017012Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:43:34.863976Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:43:40.931789Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T13:43:53.078Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 62 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "942b14e0-c49e-4013-9184-d2205ad6505d" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "10.132.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.229:46312 10.132.0.229:443 10.132.0.20:36772 - v1-completions-path 2026-11-03T13:43:54.427379Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:43:54.427426Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:43:54.427447Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:43:54.427454Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T13:43:54.075Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 57 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "553cd9e7-1de3-4b27-9a71-f4483de799e8" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "10.132.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.229:46312 10.132.0.229:443 10.132.0.20:36782 - v1-completions-path [2026-11-03T13:43:54.319Z] "DELETE /v1/api-keys/e26fef5f-aa0f-4996-96dc-324bcd9526d8 HTTP/1.1" 200 - via_upstream - "-" 0 365 107 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "4c869e93-0791-4ec8-a5f9-96057d69bc75" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "10.132.0.238:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.229:59510 10.132.0.229:443 10.132.0.20:36786 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T13:44:08.167819Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:44:14.299860Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:44:20.013304Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T13:44:36.141Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 68 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "a387a55b-0421-4a45-a526-2980e705697f" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "10.132.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.229:46312 10.132.0.229:443 10.132.0.20:40068 - v1-completions-path 2026-11-03T13:44:50.720549Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:45:01.493490Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:45:07.713770Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:45:15.265901Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:45:26.137058Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T13:45:33.478Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 63 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "f841d95f-4ca1-41cf-8265-91c26b29d4b7" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "10.132.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.229:40342 10.132.0.229:443 10.132.0.20:40426 - v1-completions-path [2026-11-03T13:45:35.612Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 78 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "41f30b51-73be-4ad9-aee3-4a17ffc7451a" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.229:443 10.132.0.20:40446 - v1-completions-path 2026-11-03T13:45:49.475874Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:46:00.307832Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:46:04.964123Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:46:23.043537Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:46:29.964293Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:46:50.317030Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:47:11.972924Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:47:33.501569Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:47:53.846205Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:48:04.824984Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:48:15.528930Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:48:26.234615Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:48:37.112794Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T13:48:47.810938Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T13:48:54.454Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 288 75 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "a436671d-bbdb-4d0a-bd86-5501dc7ad3a1" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.229:43404 10.132.0.229:443 10.132.0.20:51122 - v1-completions-path [2026-11-03T13:48:55.244Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 38 - "44.212.242.249,10.132.0.20" "python-requests/2.34.2" "3e88e0f5-fa2a-411b-8a79-95f4880ba33f" "maas.apps.3c315f49dfe0b1b71593.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.229:443 10.132.0.20:51144 - v1-completions-path