2026-10-31T02:02:29.671996Z info FLAG: --concurrency="0" 2026-10-31T02:02:29.672016Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-31T02:02:29.672020Z info FLAG: --help="false" 2026-10-31T02:02:29.672023Z info FLAG: --log_as_json="false" 2026-10-31T02:02:29.672025Z info FLAG: --log_caller="" 2026-10-31T02:02:29.672027Z info FLAG: --log_output_level="default:info" 2026-10-31T02:02:29.672029Z info FLAG: --log_stacktrace_level="default:none" 2026-10-31T02:02:29.672033Z info FLAG: --log_target="[stdout]" 2026-10-31T02:02:29.672036Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-31T02:02:29.672038Z info FLAG: --outlierLogPath="" 2026-10-31T02:02:29.672040Z info FLAG: --profiling="true" 2026-10-31T02:02:29.672042Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-31T02:02:29.672044Z info FLAG: --proxyLogLevel="warning" 2026-10-31T02:02:29.672047Z info FLAG: --serviceCluster="istio-proxy" 2026-10-31T02:02:29.672049Z info FLAG: --stsPort="0" 2026-10-31T02:02:29.672050Z info FLAG: --templateFile="" 2026-10-31T02:02:29.672052Z info FLAG: --tokenManagerPlugin="" 2026-10-31T02:02:29.672055Z info FLAG: --vklog="0" 2026-10-31T02:02:29.672058Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-31T02:02:29.672210Z info Proxy role ips=[10.132.0.208] type=router id=maas-default-gateway-openshift-default-848f6bf8bc-b5rh5.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-31T02:02:29.672255Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-31T02:02:29.673322Z info cpu limit detected as 2, setting concurrency 2026-10-31T02:02:29.673519Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-31T02:02:29.673530Z info JWT policy is third-party-jwt 2026-10-31T02:02:29.673533Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-31T02:02:29.674711Z info Starting default Istio SDS Server 2026-10-31T02:02:29.674736Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-31T02:02:29.674755Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-31T02:02:29.675060Z info Opening status port 15020 2026-10-31T02:02:29.676361Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-31T02:02:29.676803Z info sds Starting SDS grpc server 2026-10-31T02:02:29.676845Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-31T02:02:29.677548Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-31T02:02:29.678345Z info Starting proxy agent 2026-10-31T02:02:29.678381Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-31T02:02:29.746956Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-31T02:02:29.772071Z info cache generated new workload certificate resourceName=default latency=95.145183ms ttl=23h59m59.227933645s 2026-10-31T02:02:29.772114Z info cache Root cert has changed, start rotating root cert 2026-10-31T02:02:29.772158Z info cache returned workload trust anchor from cache ttl=23h59m59.22784277s 2026-10-31T02:02:29.923200Z info ads ADS: new connection for node:1 2026-10-31T02:02:29.923261Z info cache returned workload certificate from cache ttl=23h59m59.076740325s 2026-10-31T02:02:29.923759Z info ads ADS: new connection for node:2 2026-10-31T02:02:29.923844Z info cache returned workload trust anchor from cache ttl=23h59m59.076157033s 2026-10-31T02:02:30.060021Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:02:38.586746Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:02:39.276353Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-10-31T02:02:40.179020Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-10-31T02:02:41.289261Z info Readiness succeeded in 11.620421467s 2026-10-31T02:02:41.289703Z info Envoy proxy is ready 2026-10-31T02:02:50.166438Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:03:00.420385Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:03:16.745983Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:03:28.339103Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:03:31.106342Z info ads ADS: new connection for node:3 2026-10-31T02:03:31.106490Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-31T02:03:31.106513Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-31T02:03:33.372915Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:04:19.565646Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:04:43.522759Z info ads ADS: "" 3 terminated 2026-10-31T02:04:43.791277Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:04:48.623719Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:05:05.569136Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:07:41.402815Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:08:11.388145Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:08:41.323952Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:08:53.260843Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:08:59.414324Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:09:15.375555Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:09:26.530399Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:09:32.663691Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:09:52.060664Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:09:54.373303Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-31T02:09:54.373344Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-31T02:09:54.373359Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-10-31T02:09:54.373364Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 [2026-10-31T02:09:54.245Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 88 330 127 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "c877f86d-0b9e-4cd3-a916-07ababa5fb75" "maas.apps.5ce226450bd4aaa499fe.prod.konflux-ocp-ci.dev" "10.133.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.208:57292 10.132.0.208:443 10.132.0.9:35734 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T02:10:09.623Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 76 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "fce3c930-1b49-4242-b06c-a1229d78fcc3" "maas.apps.5ce226450bd4aaa499fe.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.208:55600 10.132.0.208:443 10.132.0.9:54310 - v1-completions-path [2026-10-31T02:10:24.374Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 289 73 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "9b7ea732-827b-45fd-aa6f-e8f058c24439" "maas.apps.5ce226450bd4aaa499fe.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.208:34746 10.132.0.208:443 10.132.0.9:43406 - v1-completions-path 2026-10-31T02:10:39.934671Z info ads ADS: new connection for node:4 2026-10-31T02:10:39.940093Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-10-31T02:10:39.940199Z info cache read certificate from file resource=file-root:system 2026-10-31T02:10:43.849835Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:10:44.924433Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:10:54.790717Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"163","StreamId":"7939506794141788074"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=25 [2026-10-31T02:10:54.729Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 61 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "e34989fa-75db-4b6e-ad3c-a6ad5c1e7afd" "maas.apps.5ce226450bd4aaa499fe.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.132.0.208:443 10.132.0.9:37104 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-10-31T02:12:17.341669Z info ads ADS: "" 4 terminated with stream closed 2026-10-31T02:12:17.417390Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:12:19.961579Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:12:30.486377Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T02:12:34.854Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 74 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "daf00df3-ef2e-4c18-a6ae-02160cf2c903" "maas.apps.5ce226450bd4aaa499fe.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.208:46854 10.132.0.208:443 10.132.0.9:34224 - v1-completions-path 2026-10-31T02:12:48.832087Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:12:53.599900Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:13:13.833132Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:13:30.771666Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T02:13:38.638Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 66 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "fb856437-b988-4a67-9117-45349a896f59" "maas.apps.5ce226450bd4aaa499fe.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.208:54234 10.132.0.208:443 10.132.0.9:57138 - v1-completions-path 2026-10-31T02:13:40.037022Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-10-31T02:13:39.983Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 57 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "6d4dee5b-5a28-41ce-9f4e-75ec0dd9a00d" "maas.apps.5ce226450bd4aaa499fe.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.208:46854 10.132.0.208:443 10.132.0.9:57146 - v1-completions-path 2026-10-31T02:14:01.651583Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:14:12.468800Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:14:18.841442Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:14:29.432462Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=26 2026-10-31T02:14:29.432551Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-31T02:14:29.432584Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-31T02:14:29.432609Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-10-31T02:14:29.432615Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 [2026-10-31T02:14:29.313Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 317 119 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "9ee506f5-2f1c-43e9-a66f-9efd47ba0856" "maas.apps.5ce226450bd4aaa499fe.prod.konflux-ocp-ci.dev" "10.133.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.208:50872 10.132.0.208:443 10.132.0.9:46898 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T02:14:31.999Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 58 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "fec01a26-80aa-47c2-a119-e7672509df79" "maas.apps.5ce226450bd4aaa499fe.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.208:54234 10.132.0.208:443 10.132.0.9:46912 - v1-completions-path [2026-10-31T02:14:33.538Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 56 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "f75b18e1-c8f1-447c-a8d3-5045bcdd0a65" "maas.apps.5ce226450bd4aaa499fe.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.208:54234 10.132.0.208:443 10.132.0.9:46926 - v1-completions-path 2026-10-31T02:14:47.015257Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:14:51.640476Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:15:05.163388Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:15:15.232709Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:15:25.871645Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:15:32.387515Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:15:47.115275Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:16:00.421239Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:16:06.422890Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:16:24.483222Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:16:26.658991Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T02:16:26.659026Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T02:16:26.659043Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T02:16:26.659048Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-31T02:16:26.545Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 89 331 113 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "655b31c2-99f2-4c43-98fc-98ded53c4f7f" "maas.apps.5ce226450bd4aaa499fe.prod.konflux-ocp-ci.dev" "10.133.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.208:43612 10.132.0.208:443 10.132.0.9:33128 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T02:16:29.890506Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:16:41.273676Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:17:08.096868Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:17:18.805896Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:17:22.017045Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-31T02:17:22.017084Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-31T02:17:22.017101Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-31T02:17:22.017106Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-10-31T02:17:21.895Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 121 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "cba517ee-7ade-487c-8d10-5eda0913f2f2" "maas.apps.5ce226450bd4aaa499fe.prod.konflux-ocp-ci.dev" "10.133.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.208:53090 10.132.0.208:443 10.132.0.9:37612 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T02:17:37.380038Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:17:45.119974Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:20:46.736404Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:22:15.131075Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:22:34.506156Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:22:39.479728Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:22:52.464873Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:23:09.072042Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:23:14.041811Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:23:33.820687Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:23:44.581452Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:23:50.864483Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:24:21.021598Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:24:27.291153Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:24:33.429287Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T02:24:41.462Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 61 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "90390013-2b2b-42af-98b2-a80cf339b766" "maas.apps.5ce226450bd4aaa499fe.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.208:46854 10.132.0.208:443 10.132.0.9:41284 - v1-completions-path [2026-10-31T02:24:41.824Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 60 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "4a5097ce-4535-487b-bba1-333cf7d418c9" "maas.apps.5ce226450bd4aaa499fe.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.208:54234 10.132.0.208:443 10.132.0.9:41286 - v1-completions-path 2026-10-31T02:25:02.665568Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:25:13.362633Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:25:19.949300Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:25:26.051593Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:25:31.153632Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T02:25:31.153667Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T02:25:31.153683Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T02:25:31.153688Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-31T02:25:31.033Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 120 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "33dbf9ac-4bf3-4c08-8670-8c2037896e48" "maas.apps.5ce226450bd4aaa499fe.prod.konflux-ocp-ci.dev" "10.134.1.70:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.208:44298 10.132.0.208:443 10.132.0.9:53436 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T02:25:35.276422Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T02:25:42.151Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 64 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "27526f1c-86d9-46de-a48e-c6a5f438de4f" "maas.apps.5ce226450bd4aaa499fe.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.208:46854 10.132.0.208:443 10.132.0.9:57782 - v1-completions-path 2026-10-31T02:25:58.240890Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:26:08.878612Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:26:13.812404Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:26:31.637291Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:26:39.563879Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:27:01.872179Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:27:23.796959Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:27:43.976274Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:27:54.818293Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:28:05.815462Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:28:16.649716Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:28:27.521892Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:28:38.331942Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:28:49.078684Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:28:59.694596Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T02:29:07.751154Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=26 2026-10-31T02:29:07.751245Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=26 2026-10-31T02:29:07.751279Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=26 2026-10-31T02:29:07.751303Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=26 [2026-10-31T02:29:07.724Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 401 - - "-" 66 88 26 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "716c6ec1-54b4-426f-832c-46885ff48150" "maas.apps.5ce226450bd4aaa499fe.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.208:443 10.132.0.9:45106 - v1-completions-path [2026-10-31T02:29:10.783Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 40 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "c638509f-f47f-4d7c-b57c-eaf42784f5d6" "maas.apps.5ce226450bd4aaa499fe.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.208:443 10.132.0.9:45120 - v1-completions-path