2026-10-30T07:42:35.748621Z info FLAG: --concurrency="0" 2026-10-30T07:42:35.748640Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T07:42:35.748645Z info FLAG: --help="false" 2026-10-30T07:42:35.748647Z info FLAG: --log_as_json="false" 2026-10-30T07:42:35.748650Z info FLAG: --log_caller="" 2026-10-30T07:42:35.748652Z info FLAG: --log_output_level="default:info" 2026-10-30T07:42:35.748654Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T07:42:35.748659Z info FLAG: --log_target="[stdout]" 2026-10-30T07:42:35.748661Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T07:42:35.748663Z info FLAG: --outlierLogPath="" 2026-10-30T07:42:35.748665Z info FLAG: --profiling="true" 2026-10-30T07:42:35.748667Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T07:42:35.748671Z info FLAG: --proxyLogLevel="warning" 2026-10-30T07:42:35.748673Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T07:42:35.748676Z info FLAG: --stsPort="0" 2026-10-30T07:42:35.748678Z info FLAG: --templateFile="" 2026-10-30T07:42:35.748680Z info FLAG: --tokenManagerPlugin="" 2026-10-30T07:42:35.748683Z info FLAG: --vklog="0" 2026-10-30T07:42:35.748685Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T07:42:35.748845Z info Proxy role ips=[10.132.0.189] type=router id=maas-default-gateway-openshift-default-67bf986f99-p9f76.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T07:42:35.748901Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T07:42:35.750179Z info cpu limit detected as 2, setting concurrency 2026-10-30T07:42:35.750411Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T07:42:35.750423Z info JWT policy is third-party-jwt 2026-10-30T07:42:35.750426Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T07:42:35.752359Z info Starting default Istio SDS Server 2026-10-30T07:42:35.752383Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T07:42:35.752410Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T07:42:35.753165Z info Opening status port 15020 2026-10-30T07:42:35.753810Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T07:42:35.754824Z info sds Starting SDS grpc server 2026-10-30T07:42:35.754906Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T07:42:35.755569Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T07:42:35.756460Z info Starting proxy agent 2026-10-30T07:42:35.756512Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T07:42:35.822800Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T07:42:35.840889Z info cache generated new workload certificate resourceName=default latency=85.891142ms ttl=23h59m59.159114285s 2026-10-30T07:42:35.840926Z info cache Root cert has changed, start rotating root cert 2026-10-30T07:42:35.840965Z info cache returned workload trust anchor from cache ttl=23h59m59.159035884s 2026-10-30T07:42:35.934537Z info ads ADS: new connection for node:1 2026-10-30T07:42:35.934644Z info cache returned workload certificate from cache ttl=23h59m59.065358543s 2026-10-30T07:42:35.935050Z info ads ADS: new connection for node:2 2026-10-30T07:42:35.935149Z info cache returned workload trust anchor from cache ttl=23h59m59.06485195s 2026-10-30T07:42:36.017341Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:42:43.965832Z info Readiness succeeded in 8.22031074s 2026-10-30T07:42:43.966266Z info Envoy proxy is ready 2026-10-30T07:43:05.328137Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:43:22.141194Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:43:25.101957Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:43:35.449807Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:43:53.559473Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:43:55.902501Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:44:10.628660Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:44:21.675221Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:44:24.158548Z info ads ADS: new connection for node:3 2026-10-30T07:44:24.158787Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T07:44:24.158816Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T07:44:28.048543Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:45:06.622911Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:45:09.440891Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T07:45:09.440933Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T07:45:09.440949Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T07:45:09.440954Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T07:45:07.459Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 1981 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "a764f1f6-63cd-47a1-bd4d-63ec68a44ef6" "maas.apps.59c8f9f7001e17a9d6fe.prod.konflux-ocp-ci.dev" "10.132.0.53:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.189:55618 10.132.0.189:443 10.132.0.11:33290 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T07:45:26.180984Z info ads ADS: "" 3 terminated with stream closed 2026-10-30T07:45:26.250676Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:45:28.566852Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T07:45:30.421Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 283 75 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "44e23fb2-011b-44dc-971a-1d767dc8ae8c" "maas.apps.59c8f9f7001e17a9d6fe.prod.konflux-ocp-ci.dev" "10.134.0.38:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.189:40766 10.132.0.189:443 10.132.0.11:41120 - v1-completions-path 2026-10-30T07:45:31.429221Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:45:36.823926Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:47:13.108266Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:47:29.711174Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:47:45.449696Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:47:57.451814Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:48:01.603875Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:48:12.767758Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:48:17.178106Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:48:23.196738Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T07:48:23.196780Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T07:48:23.196799Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T07:48:23.196806Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T07:48:23.115Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 335 80 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "3cc373df-725c-4daa-9cd2-86e520fc445e" "maas.apps.59c8f9f7001e17a9d6fe.prod.konflux-ocp-ci.dev" "10.132.0.53:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.189:35716 10.132.0.189:443 10.132.0.11:43928 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T07:48:35.680336Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:49:26.361169Z info ads ADS: new connection for node:4 2026-10-30T07:49:26.368426Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-10-30T07:49:26.368567Z info cache read certificate from file resource=file-root:system 2026-10-30T07:49:27.581574Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:49:30.087054Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:49:54.609487Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:49:54.609527Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:49:54.609539Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:49:54.609543Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T07:49:54.609733Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T07:49:54.483Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1723 126 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "0852745f-d27f-4e3c-83c6-692c6312039b" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.189:48236 10.132.0.189:443 10.132.0.11:40732 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T07:50:24.459455Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:50:24.459488Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:50:24.459499Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:50:24.459503Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T07:50:24.459683Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T07:50:24.360Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1730 99 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "05a69fc1-c9b7-467c-a943-62957aae657d" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.189:48236 10.132.0.189:443 10.132.0.11:60060 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T07:50:26.583079Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:50:26.583117Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:50:26.583127Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:50:26.583132Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T07:50:26.583315Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T07:50:26.486Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1728 97 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "465b1152-ab81-4448-8b88-da8b2d78e54e" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.189:48236 10.132.0.189:443 10.132.0.11:60074 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T07:50:55.155860Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:50:55.287030Z info ads ADS: "" 4 terminated 2026-10-30T07:50:55.735948Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:50:58.931211Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:51:04.148148Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T07:51:11.487Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 80 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "b68ec80e-a397-4cab-8762-e59bdf360f3e" "maas.apps.59c8f9f7001e17a9d6fe.prod.konflux-ocp-ci.dev" "10.133.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.189:41914 10.132.0.189:443 10.132.0.11:40588 - v1-completions-path 2026-10-30T07:51:24.014150Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:51:28.323702Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T07:51:32.035Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 59 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "71cc0645-d7c0-4f95-9292-f3318e637227" "maas.apps.59c8f9f7001e17a9d6fe.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.189:443 10.132.0.11:49306 - v1-completions-path 2026-10-30T07:51:44.231810Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:52:00.112353Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:52:05.670178Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-10-30T07:52:05.616Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 60 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "91bde89c-0b41-419f-b114-04bf57c453dc" "maas.apps.59c8f9f7001e17a9d6fe.prod.konflux-ocp-ci.dev" "10.133.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.189:41914 10.132.0.189:443 10.132.0.11:59020 - v1-completions-path [2026-10-30T07:52:06.440Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 59 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "b6039a32-3346-459a-abdf-3c337ce85e92" "maas.apps.59c8f9f7001e17a9d6fe.prod.konflux-ocp-ci.dev" "10.133.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.189:41914 10.132.0.189:443 10.132.0.11:59026 - v1-completions-path 2026-10-30T07:52:23.655539Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:52:34.011721Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:52:38.695888Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:52:43.099696Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-10-30T07:52:43.036Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 63 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "37a85091-e9a3-4355-92ab-4dcd2e64cd9b" "maas.apps.59c8f9f7001e17a9d6fe.prod.konflux-ocp-ci.dev" "10.133.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.189:47880 10.132.0.189:443 10.132.0.11:48536 - v1-completions-path 2026-10-30T07:52:55.700518Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:52:57.964618Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:53:17.076158Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:53:21.675058Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:53:37.261736Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:53:39.599710Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:53:57.487586Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:54:09.150211Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:54:12.979603Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:54:34.555311Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T07:54:34.555348Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T07:54:34.555361Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T07:54:34.555367Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T07:54:34.480Z] "DELETE /v1/api-keys/cb314acc-c892-4667-8b76-c39540fd31f7 HTTP/1.1" 404 - via_upstream - "-" 0 29 75 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "84f2e792-3962-4f65-a34f-06d527b8276e" "maas.apps.59c8f9f7001e17a9d6fe.prod.konflux-ocp-ci.dev" "10.132.0.53:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.189:59288 10.132.0.189:443 10.132.0.11:56418 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T07:54:41.252113Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:54:59.390713Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:55:09.776290Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:57:11.553285Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:57:15.210873Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:58:46.457673Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:59:02.338090Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:59:04.746827Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:59:23.699648Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:59:36.053657Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:59:39.430906Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:59:55.520651Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:00:06.091558Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:00:11.154418Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T08:00:15.360Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 64 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "01485266-274f-4dc1-96a2-93c7572583f8" "maas.apps.59c8f9f7001e17a9d6fe.prod.konflux-ocp-ci.dev" "10.133.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.189:47880 10.132.0.189:443 10.132.0.11:46516 - v1-completions-path 2026-10-30T08:00:15.983170Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T08:00:15.983212Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T08:00:15.983228Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T08:00:15.983233Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T08:00:15.906Z] "DELETE /v1/api-keys/d66ae90b-d873-42bd-bf87-98bdb2d4b167 HTTP/1.1" 200 - via_upstream - "-" 0 365 76 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "80591ab8-2266-47e5-b48e-59377d5e1683" "maas.apps.59c8f9f7001e17a9d6fe.prod.konflux-ocp-ci.dev" "10.134.0.103:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.189:40244 10.132.0.189:443 10.132.0.11:46526 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T08:00:28.030178Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:00:32.731866Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:00:35.280419Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T08:00:42.210Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 63 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "911d1edb-c8d4-4f72-9df1-7fa956770534" "maas.apps.59c8f9f7001e17a9d6fe.prod.konflux-ocp-ci.dev" "10.133.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.189:47880 10.132.0.189:443 10.132.0.11:39046 - v1-completions-path [2026-10-30T08:00:42.477Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 60 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "2f6ae2a9-2e3b-44da-8234-e951b3a97cad" "maas.apps.59c8f9f7001e17a9d6fe.prod.konflux-ocp-ci.dev" "10.133.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.189:47880 10.132.0.189:443 10.132.0.11:39052 - v1-completions-path [2026-10-30T08:00:43.286Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 60 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "a2104ccf-7da7-4dbd-996d-08f2645cd79a" "maas.apps.59c8f9f7001e17a9d6fe.prod.konflux-ocp-ci.dev" "10.133.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.189:41914 10.132.0.189:443 10.132.0.11:39066 - v1-completions-path [2026-10-30T08:00:44.200Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 61 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "906713ff-019e-498e-aa1d-a1f0b0b044f1" "maas.apps.59c8f9f7001e17a9d6fe.prod.konflux-ocp-ci.dev" "10.133.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.189:41914 10.132.0.189:443 10.132.0.11:38974 - v1-completions-path [2026-10-30T08:00:46.210Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 72 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "87c2e55a-5fbd-41fc-966d-7b34277afc32" "maas.apps.59c8f9f7001e17a9d6fe.prod.konflux-ocp-ci.dev" "10.133.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.189:47880 10.132.0.189:443 10.132.0.11:38994 - v1-completions-path [2026-10-30T08:00:46.588Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 58 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "e7c474b3-e7a7-468b-a885-814c740a58de" "maas.apps.59c8f9f7001e17a9d6fe.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.189:443 10.132.0.11:39008 - v1-completions-path 2026-10-30T08:00:58.847515Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:01:09.168164Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:01:13.739788Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:01:17.171824Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:01:22.093488Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:01:40.379515Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:01:50.852169Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:01:53.192589Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:02:03.354823Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:02:08.125362Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:02:24.792972Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:02:44.683158Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:02:55.082982Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:03:05.615130Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:03:15.959139Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:03:26.286548Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:03:36.648534Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:03:47.005149Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:03:57.238632Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:04:07.616632Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T08:04:18.145956Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T08:04:27.321Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 46 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "1810272b-ef5d-420d-b49d-3127a485f191" "maas.apps.59c8f9f7001e17a9d6fe.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.189:443 10.132.0.11:35828 - v1-completions-path