2026-11-01T08:43:01.338500Z info FLAG: --concurrency="0" 2026-11-01T08:43:01.338519Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-01T08:43:01.338523Z info FLAG: --help="false" 2026-11-01T08:43:01.338526Z info FLAG: --log_as_json="false" 2026-11-01T08:43:01.338528Z info FLAG: --log_caller="" 2026-11-01T08:43:01.338530Z info FLAG: --log_output_level="default:info" 2026-11-01T08:43:01.338532Z info FLAG: --log_stacktrace_level="default:none" 2026-11-01T08:43:01.338544Z info FLAG: --log_target="[stdout]" 2026-11-01T08:43:01.338546Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-01T08:43:01.338548Z info FLAG: --outlierLogPath="" 2026-11-01T08:43:01.338550Z info FLAG: --profiling="true" 2026-11-01T08:43:01.338552Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-01T08:43:01.338554Z info FLAG: --proxyLogLevel="warning" 2026-11-01T08:43:01.338556Z info FLAG: --serviceCluster="istio-proxy" 2026-11-01T08:43:01.338558Z info FLAG: --stsPort="0" 2026-11-01T08:43:01.338560Z info FLAG: --templateFile="" 2026-11-01T08:43:01.338562Z info FLAG: --tokenManagerPlugin="" 2026-11-01T08:43:01.338568Z info FLAG: --vklog="0" 2026-11-01T08:43:01.338571Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-01T08:43:01.338819Z info Proxy role ips=[10.134.0.42] type=router id=maas-default-gateway-openshift-default-746ff8f49f-gsql5.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-01T08:43:01.338870Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-01T08:43:01.340253Z info cpu limit detected as 2, setting concurrency 2026-11-01T08:43:01.340468Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-01T08:43:01.340480Z info JWT policy is third-party-jwt 2026-11-01T08:43:01.340483Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-01T08:43:01.342284Z info Starting default Istio SDS Server 2026-11-01T08:43:01.342313Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-01T08:43:01.342345Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-01T08:43:01.342960Z info Opening status port 15020 2026-11-01T08:43:01.344549Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-01T08:43:01.346191Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-01T08:43:01.346982Z info Starting proxy agent 2026-11-01T08:43:01.347015Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-01T08:43:01.354303Z info sds Starting SDS grpc server 2026-11-01T08:43:01.354327Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-01T08:43:01.413907Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-01T08:43:01.477741Z info cache generated new workload certificate resourceName=default latency=132.858009ms ttl=23h59m59.522265609s 2026-11-01T08:43:01.477786Z info cache Root cert has changed, start rotating root cert 2026-11-01T08:43:01.477823Z info cache returned workload trust anchor from cache ttl=23h59m59.522177917s 2026-11-01T08:43:01.596921Z info ads ADS: new connection for node:1 2026-11-01T08:43:01.597004Z info cache returned workload certificate from cache ttl=23h59m59.402998589s 2026-11-01T08:43:01.597331Z info ads ADS: new connection for node:2 2026-11-01T08:43:01.597426Z info cache returned workload trust anchor from cache ttl=23h59m59.402575895s 2026-11-01T08:43:01.686453Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T08:43:11.608694Z info Readiness succeeded in 10.272936784s 2026-11-01T08:43:11.609157Z info Envoy proxy is ready [2026-11-01T08:43:29.775Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 67 - "52.71.36.72,10.133.0.18" "curl/7.76.1" "b75373c4-1f6e-4ad9-aa8b-905c38da1245" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.42:443 10.133.0.18:42048 - v1-model-routing 2026-11-01T08:43:32.965929Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T08:43:33.269992Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-01T08:43:32.949Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 91 16 - "52.71.36.72,10.133.0.18" "curl/7.76.1" "df990fb2-ffbc-4b50-a176-6e204b003f79" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.133.0.52:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.42:44608 10.134.0.42:443 10.133.0.18:42078 - odh-ai-gateway-infra.maas-api-route.4 [2026-11-01T08:43:33.165Z] "GET /v1/subscriptions HTTP/1.1" 200 - via_upstream - "-" 0 488 104 - "52.71.36.72,10.133.0.18" "curl/7.76.1" "c34b0429-7e1c-4d9d-9e12-193309c4e8d0" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.133.0.52:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.42:44608 10.134.0.42:443 10.133.0.18:42090 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T08:49:16.190866Z info ads ADS: new connection for node:3 2026-11-01T08:49:16.198405Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-01T08:49:16.198465Z info cache read certificate from file resource=file-root:system 2026-11-01T08:49:21.384696Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T08:49:25.233934Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T08:49:36.737810Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T08:49:49.820562Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T08:50:03.861783Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-01T08:50:03.862084Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-11-01T08:50:03.772Z] "POST /llm/e2e-rt-sw-14c1a4/anything/e2e-rt-sw-14c1a4-client HTTP/1.1" 503 UH no_healthy_upstream - "-" 124 19 90 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "1cf850aa-accb-4f68-9cc7-9d52b54f5661" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rt-sw-14c1a4.llm.svc.cluster.local - 10.134.0.42:443 10.133.0.18:59850 - llm.external-model-e2e-rt-sw-14c1a4-ee78529a70e4.2 2026-11-01T08:50:27.546424Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T08:50:43.853861Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T08:50:43.854235Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-01T08:50:43.734Z] "POST /llm/e2e-rt-sw-14c1a4/anything/e2e-rt-sw-14c1a4-client HTTP/1.1" 503 UH no_healthy_upstream - "-" 124 19 119 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "51a9121f-86b4-40c9-9f7d-f3aeebda2672" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rt-sw-14c1a4.llm.svc.cluster.local - 10.134.0.42:443 10.133.0.18:60346 - llm.external-model-e2e-rt-sw-14c1a4-ee78529a70e4.2 2026-11-01T08:51:14.295004Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T08:51:26.560767Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T08:51:42.824Z] "POST /llm/e2e-rt-eq-c1177c/anything/e2e-rt-eq-c1177c-client HTTP/1.1" 503 - - "-" 124 0 114 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "ac1609a5-c03b-4c89-aeff-3932f9f1c4c8" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rt-eq-c1177c.llm.svc.cluster.local - 10.134.0.42:443 10.133.0.18:58054 - llm.external-model-e2e-rt-eq-c1177c-8efa217782bd.4 [2026-11-01T08:51:53.528Z] "POST /llm/e2e-rt-eq-c1177c/anything/e2e-rt-eq-c1177c-client HTTP/1.1" 503 - - "-" 124 0 70 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "13de5785-8a98-4500-96ea-0b83051433a6" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rt-eq-c1177c.llm.svc.cluster.local - 10.134.0.42:443 10.133.0.18:47616 - llm.external-model-e2e-rt-eq-c1177c-8efa217782bd.4 [2026-11-01T08:52:25.381Z] "POST /llm/e2e-rt-eq-c1177c/anything/e2e-rt-eq-c1177c-client HTTP/1.1" 503 - - "-" 124 0 84 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "ea7c19bb-972c-4fe2-95a7-362a8fa2913e" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rt-eq-c1177c.llm.svc.cluster.local - 10.134.0.42:443 10.133.0.18:60174 - llm.external-model-e2e-rt-eq-c1177c-8efa217782bd.4 [2026-11-01T08:54:21.479Z] "POST /llm/e2e-rt-eq-c1177c/anything/e2e-rt-eq-c1177c-client HTTP/1.1" 503 - - "-" 124 0 70 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "1693f086-6ac1-46a1-8e71-91551cadf09a" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rt-eq-c1177c.llm.svc.cluster.local - 10.134.0.42:443 10.133.0.18:42416 - llm.external-model-e2e-rt-eq-c1177c-8efa217782bd.4 [2026-11-01T08:54:44.724Z] "POST /llm/e2e-rt-eq-c1177c/anything/e2e-rt-eq-c1177c-client HTTP/1.1" 503 - - "-" 124 0 77 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "3a6fe910-42f8-479e-850b-8081a5895440" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rt-eq-c1177c.llm.svc.cluster.local - 10.134.0.42:443 10.133.0.18:33156 - llm.external-model-e2e-rt-eq-c1177c-8efa217782bd.4 2026-11-01T08:54:53.479130Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T08:55:17.504179Z info ads ADS: "" 3 terminated with stream closed 2026-11-01T08:55:18.726361Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T08:55:27.757094Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T08:56:40.270315Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T09:00:13.302Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 89 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "d16bd618-1071-4d54-9de2-0040d3770ec8" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.134.0.42:443 10.133.0.18:41392 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T09:00:15.873026Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-01T09:00:14.495Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 1377 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "99776a12-bc1e-4ca6-9f5f-05532ef62636" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.133.0.52:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.42:34786 10.134.0.42:443 10.133.0.18:41404 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T09:00:17.684884Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-01T09:00:17.568Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 116 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "d3d3b289-9feb-4942-a49e-750ab7d7e788" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.133.0.52:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.42:34800 10.134.0.42:443 10.133.0.18:42244 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T09:00:58.778Z] "POST /maas-api/internal/v1/subscriptions/select HTTP/1.1" 404 NR route_not_found - "-" 56 0 10 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "5da78565-6ed7-48eb-bef3-5fb1f7f6bef0" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" - - 10.134.0.42:443 10.133.0.18:49604 - - [2026-11-01T09:00:59.505Z] "POST /maas-api/internal/v1/api-keys/validate HTTP/1.1" 404 NR route_not_found - "-" 31 0 7 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "c563faf3-f2cc-4cd3-af35-a49f30f30406" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" - - 10.134.0.42:443 10.133.0.18:49618 - - [2026-11-01T09:09:09.348Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 110 318 96 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "a4bacac0-5ff2-44ed-98f6-a502c946266f" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.42:46628 10.134.0.42:443 10.133.0.18:53094 - v1-chat-completions-path 2026-11-01T09:13:18.614412Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-01T09:15:06.464640Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T09:15:06.829920Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-01T09:15:06.339Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 125 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "c6eeed5f-7cca-4296-8159-d15a39e039ca" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.133.0.52:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.42:55076 10.134.0.42:443 10.133.0.18:50020 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T09:15:06.709Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 86 329 120 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "8ceee2a9-e9c9-4b73-831e-9c12de73ae21" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.133.0.52:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.42:55076 10.134.0.42:443 10.133.0.18:50022 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T09:15:07.431496Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=26 2026-11-01T09:15:07.685682Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T09:15:07.685958Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 2026-11-01T09:15:07.686032Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-01T09:15:07.686040Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-01T09:15:07.686043Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-01T09:15:07.686047Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-11-01T09:15:07.089Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 136 346 78 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "c0382f76-0c15-45eb-a762-8914d2e68c7d" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.42:46628 10.134.0.42:443 10.133.0.18:50030 - v1-chat-completions-path [2026-11-01T09:15:07.372Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 135 1554 66 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "36a878b8-720a-4b0b-9c69-0efe4ecf9285" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.42:46628 10.134.0.42:443 10.133.0.18:50038 - v1-chat-completions-path [2026-11-01T09:15:07.623Z] "POST /v1/completions HTTP/1.1" 404 - via_upstream - "-" 94 126 62 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "a7bfa10c-f3a4-4c51-ba5a-fe702af937c1" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.42:48914 10.134.0.42:443 10.133.0.18:50040 - v1-completions-path 2026-11-01T09:19:40.904465Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-01T09:19:40.904776Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 2026-11-01T09:19:40.904862Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=26 2026-11-01T09:19:40.904873Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=26 2026-11-01T09:19:40.904878Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=26 2026-11-01T09:19:40.904884Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=26 [2026-11-01T09:19:40.255Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 113 318 91 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "25b7a8ca-50a3-439d-a1d8-b9fd6f8054a7" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.42:48914 10.134.0.42:443 10.133.0.18:55584 - v1-chat-completions-path [2026-11-01T09:19:40.838Z] "POST /v1/chat/completions HTTP/1.1" 404 - via_upstream - "-" 126 129 66 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "7c0fb714-2f3c-40eb-bb3a-3ef21b427712" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.42:46628 10.134.0.42:443 10.133.0.18:55592 - v1-chat-completions-path 2026-11-01T09:22:18.226067Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-01T09:22:18.212Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 91 13 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "a954e7d3-2b5c-43d9-bab6-f6f3eb32830b" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.133.0.52:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.42:48336 10.134.0.42:443 10.133.0.18:42134 - odh-ai-gateway-infra.maas-api-route.4 [2026-11-01T09:28:52.957Z] "GET /v1/tenants HTTP/1.1" 404 NR route_not_found - "-" 0 0 9 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "b1f8346e-dae6-4f80-a886-5fba2f3302eb" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" - - 10.134.0.42:443 10.133.0.18:33252 - - 2026-11-01T09:37:06.097022Z info ads ADS: new connection for node:4 2026-11-01T09:37:06.102460Z info cache read certificate from file resource=file-root:system 2026-11-01T09:37:09.080891Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T09:37:31.219259Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T09:37:40.011144Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-01T09:37:40.029526Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-01T09:37:39.843Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 66 309 186 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "1dcafb29-d0a2-4b71-91fc-3760b820d11e" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.133.0.52:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.42:44774 10.134.0.42:443 10.133.0.18:60496 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T09:37:40.558027Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-01T09:37:40.558278Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-11-01T09:37:40.289Z] "POST /anything/e2e-dft-1e1942-07a051 HTTP/1.1" 200 - via_upstream - "-" 109 1762 268 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "364641d6-1a67-4026-b07b-6872839c1e0c" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.42:34504 10.134.0.42:443 10.133.0.18:60510 - llm.external-model-e2e-dft-1e1942-37a955620972.0 2026-11-01T09:37:41.361423Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T09:37:41.361653Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-01T09:37:41.141Z] "POST /anything/e2e-dft-1e1942-07a051 HTTP/1.1" 200 - via_upstream - "-" 109 1769 220 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "bda0f6aa-d484-4683-ad91-49dbc22b170b" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.42:34506 10.134.0.42:443 10.133.0.18:60522 - llm.external-model-e2e-dft-1e1942-37a955620972.0 2026-11-01T09:38:16.330700Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T09:39:46.230191Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T09:39:54.416Z] "POST /llm/e2e-dft-e6b37a/anything/e2e-dft-e6b37a-89e335 HTTP/1.1" 503 - - "-" 109 0 83 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "c25db637-19ea-42a3-bd80-ac4364e59f88" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-e6b37a.llm.svc.cluster.local - 10.134.0.42:443 10.133.0.18:40672 - llm.external-model-e2e-dft-e6b37a-d981b1d8aec4.2 [2026-11-01T09:41:52.006Z] "POST /llm/e2e-dft-e6b37a/anything/e2e-dft-e6b37a-89e335 HTTP/1.1" 503 - - "-" 109 0 69 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "5643c916-69ee-4873-811f-859c3d591f77" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-e6b37a.llm.svc.cluster.local - 10.134.0.42:443 10.133.0.18:53580 - llm.external-model-e2e-dft-e6b37a-d981b1d8aec4.2 [2026-11-01T09:41:57.423Z] "POST /llm/e2e-dft-e6b37a/anything/e2e-dft-e6b37a-89e335 HTTP/1.1" 503 - - "-" 109 0 65 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "21f53c31-5675-4a5d-83c6-339198edf6f0" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-e6b37a.llm.svc.cluster.local - 10.134.0.42:443 10.133.0.18:33560 - llm.external-model-e2e-dft-e6b37a-d981b1d8aec4.2 2026-11-01T09:42:01.774411Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=3 [2026-11-01T09:42:29.216Z] "POST /llm/e2e-dft-e6b37a/anything/e2e-dft-e6b37a-89e335 HTTP/1.1" 503 - - "-" 109 0 68 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "362b6159-de9a-49e5-9db2-f51ef90be595" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-e6b37a.llm.svc.cluster.local - 10.134.0.42:443 10.133.0.18:38764 - llm.external-model-e2e-dft-e6b37a-d981b1d8aec4.2 [2026-11-01T09:43:27.391Z] "POST /llm/e2e-dft-e6b37a/anything/e2e-dft-e6b37a-89e335 HTTP/1.1" 503 - - "-" 109 0 86 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "c5eb5a10-d7b9-4888-9991-7dc1ce170582" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-e6b37a.llm.svc.cluster.local - 10.134.0.42:443 10.133.0.18:39668 - llm.external-model-e2e-dft-e6b37a-d981b1d8aec4.2 2026-11-01T09:45:10.484796Z info ads ADS: "" 4 terminated 2026-11-01T09:45:11.363758Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T09:45:25.557279Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T09:46:08.697938Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T09:49:36.570026Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T09:51:51.501379Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:03:56.528834Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-01T10:03:56.416Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 112 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "783993fc-e451-440a-a350-240004d199f1" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.133.0.52:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.42:42630 10.134.0.42:443 10.133.0.18:33916 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T10:03:58.871049Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:04:24.419123Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:04:34.436956Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-01T10:04:34.327Z] "POST /v1/api-keys HTTP/1.1" 400 - via_upstream - "-" 80 94 109 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "659f8605-7678-4bf1-9c60-cf87425e626f" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.133.0.52:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.42:52278 10.134.0.42:443 10.133.0.18:56126 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T10:04:59.953054Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T10:05:03.078Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 147 412 132 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "f1c97365-6aa4-4887-8ce7-4c770cfbd728" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.132.0.31:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.134.0.42:46176 10.134.0.42:443 10.133.0.18:46464 - v1-chat-completions-path [2026-11-01T10:05:03.426Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 2817 3089 66 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "460f257b-a947-4967-8e88-c1d077033a25" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.132.0.31:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.134.0.42:46186 10.134.0.42:443 10.133.0.18:46466 - v1-chat-completions-path 2026-11-01T10:05:18.088609Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-01T10:05:18.088636Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-01T10:05:18.088639Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-01T10:05:18.088643Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-11-01T10:05:17.949Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 403 - - "-" 61 103 138 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "1c6067d6-2754-97fa-8346-77d9365b9617" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.42:443 10.133.0.18:58082 - v1-catch-all-path 2026-11-01T10:05:19.295874Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:05:23.967228Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:05:33.960148Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:05:45.320329Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:05:55.033787Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:06:12.727974Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:06:24.392523Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:06:27.031708Z info ads ADS: new connection for node:5 2026-11-01T10:06:27.031925Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-01T10:06:27.031956Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-01T10:06:31.186021Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:07:10.227325Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:07:14.279427Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-01T10:07:14.115Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 163 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "5fa63b88-7798-4ba9-ab8f-38f7c5d18d05" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.133.0.52:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.42:53194 10.134.0.42:443 10.133.0.18:36210 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T10:07:15.523Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 348 90 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "826d0bf2-137a-4c7a-ba22-b903bd4cd90a" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.132.1.120:8000" outbound|54321||e2e-prio-pool-d609fa82-inference-pool-ip-949aac53.llm.svc.cluster.local 10.134.0.42:49162 10.134.0.42:443 10.133.0.18:54244 - v1-model-routing 2026-11-01T10:07:35.531320Z info ads ADS: "" 5 terminated 2026-11-01T10:07:35.589609Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:07:41.931337Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:07:54.498189Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:10:09.715774Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-01T10:10:09.602Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 113 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "95ff2f65-6b96-4a14-aeb2-baf57898ed26" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.133.0.52:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.42:45158 10.134.0.42:443 10.133.0.18:56322 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T10:10:31.407495Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:10:58.895799Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:11:23.517387Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=4 2026-11-01T10:11:29.070322Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:11:42.694908Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:11:48.689300Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:11:53.856406Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:12:06.807228Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:12:14.363209Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:12:33.148529Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:13:26.480860Z info ads ADS: new connection for node:6 2026-11-01T10:13:26.485617Z info cache read certificate from file resource=file-root:system 2026-11-01T10:13:30.446682Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:13:31.222236Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:13:46.609470Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-01T10:13:46.609803Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-11-01T10:13:46.412Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1717 197 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "89c4f010-a5e9-4428-9449-69333d2dedd0" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.42:55914 10.134.0.42:443 10.133.0.18:47062 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-01T10:14:21.364553Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T10:14:21.364873Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-01T10:14:21.168Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1719 196 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "d68d71ce-346e-478c-9907-851fcd927cec" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.42:54140 10.134.0.42:443 10.133.0.18:59824 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-01T10:14:56.739322Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T10:14:56.739606Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-01T10:14:56.610Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1720 129 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "dee8e874-cf18-4c1e-838f-dcb7f77c717b" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.42:54140 10.134.0.42:443 10.133.0.18:41256 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-01T10:15:18.137315Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-01T10:15:18.016Z] "DELETE /v1/api-keys/469ce1b0-b333-4782-99df-7742732beb0e HTTP/1.1" 200 - via_upstream - "-" 0 359 121 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "b646a783-a486-4f5d-9975-6d575632144a" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.133.0.52:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.42:54892 10.134.0.42:443 10.133.0.18:54864 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T10:15:31.401949Z info ads ADS: "" 6 terminated with stream closed 2026-11-01T10:15:31.444699Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:15:31.983042Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=26 2026-11-01T10:15:31.983069Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=26 2026-11-01T10:15:31.983072Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=26 2026-11-01T10:15:31.983077Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=26 [2026-11-01T10:15:31.946Z] "POST /llm/premium-simulated-simulated-premium/v1/completions HTTP/1.1" 403 - - "-" 74 77 36 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "e105f05c-4e43-4020-b296-fd8eceba5246" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.42:443 10.133.0.18:38094 - v1-completions-path 2026-11-01T10:15:33.923216Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:15:36.510093Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=26 2026-11-01T10:15:36.510121Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=26 2026-11-01T10:15:36.510124Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=26 2026-11-01T10:15:36.510128Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=26 [2026-11-01T10:15:36.474Z] "POST /llm/premium-simulated-simulated-premium/v1/completions HTTP/1.1" 403 - - "-" 74 77 36 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "ec0191e8-2344-402c-a2c7-dd293268f70d" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.42:443 10.133.0.18:38122 - v1-completions-path 2026-11-01T10:15:44.234405Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T10:15:50.089Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 69 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "c3062c97-5d16-4d88-b8ed-ab89acfa4846" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.42:443 10.133.0.18:35176 - v1-completions-path 2026-11-01T10:16:03.484615Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:16:09.530936Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:16:31.360477Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:16:44.781127Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T10:16:54.132Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 288 79 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "c2611f81-f9dc-43b1-8a7a-85e7caff15dd" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.42:44424 10.134.0.42:443 10.133.0.18:57366 - v1-completions-path [2026-11-01T10:16:55.060Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 64 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "ce92917d-1713-42c2-9421-e2c5baf8d302" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.42:44428 10.134.0.42:443 10.133.0.18:57376 - v1-completions-path [2026-11-01T10:16:56.086Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 61 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "316dc65a-3bf0-4d83-9482-f134199faf32" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.42:44424 10.134.0.42:443 10.133.0.18:60558 - v1-completions-path [2026-11-01T10:16:56.609Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 72 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "e9bbca0f-6679-4954-a59a-a3340f75953a" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.42:44428 10.134.0.42:443 10.133.0.18:60560 - v1-completions-path 2026-11-01T10:17:15.876580Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:17:26.511334Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:17:32.656749Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T10:17:47.830Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 64 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "1539b063-4d4d-4559-b778-699ca299e65a" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.42:44428 10.134.0.42:443 10.133.0.18:36942 - v1-completions-path 2026-11-01T10:18:01.209891Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:18:05.732772Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:18:18.614118Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:18:27.241772Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:18:36.321163Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T10:18:38.529Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 69 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "8c450367-6416-4c91-9e08-8d3681f9cd47" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.42:46628 10.134.0.42:443 10.133.0.18:37294 - v1-completions-path 2026-11-01T10:18:44.996783Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:18:54.020515Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T10:18:56.235Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 67 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "b9dc43ac-8b73-4671-9c12-5bbc62b316ec" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.42:48914 10.134.0.42:443 10.133.0.18:58178 - v1-completions-path 2026-11-01T10:19:11.488209Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:19:17.297300Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:19:30.227139Z info cache event for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt : REMOVE, pushing to proxy 2026-11-01T10:19:36.191857Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T10:19:38.746Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 66 18 64 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "fc631121-5be2-47d2-8ef7-0801b1391e2f" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.42:443 10.133.0.18:57996 - v1-completions-path 2026-11-01T10:19:41.421157Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:19:53.886657Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:20:13.700694Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:20:24.400625Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:20:42.819781Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:20:50.290298Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:22:55.491833Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:24:25.131411Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T10:24:25.806Z] "POST /llm/e2e-trlp-test-simulated/v1/completions HTTP/1.1" 429 - - "-" 73 18 90 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "fb967565-41de-4697-a3d0-baef57946e0b" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-trlp-test-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.42:443 10.133.0.18:47276 - v1-completions-path 2026-11-01T10:24:31.980230Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-01T10:24:31.871Z] "DELETE /v1/api-keys/e2e-trlp-test-key HTTP/1.1" 404 - via_upstream - "-" 0 29 109 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "4fbe38f9-9e35-4210-9616-1acb5966a507" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.133.0.52:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.42:48806 10.134.0.42:443 10.133.0.18:47298 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T10:24:45.544225Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:24:50.678448Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:25:04.265267Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:25:18.045613Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:25:23.194048Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T10:25:28.679Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 72 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "99926bca-303a-438c-b841-a488572aed4b" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.42:44424 10.134.0.42:443 10.133.0.18:35888 - v1-completions-path 2026-11-01T10:25:44.090208Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:25:54.752222Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:26:01.336089Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T10:26:15.008Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 73 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "ad0eaf30-f3d2-4c38-8daa-9154acbb8852" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.42:44424 10.134.0.42:443 10.133.0.18:38798 - v1-completions-path 2026-11-01T10:26:31.528709Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:26:38.148942Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:26:44.310371Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T10:26:53.141Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 70 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "1fd2aaf4-879d-9ed8-b90e-4101dba62213" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.42:44424 10.134.0.42:443 10.133.0.18:55932 - v1-completions-path [2026-11-01T10:27:00.465Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 59 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "29bb86f7-10f5-475c-bdf9-62c61e20d688" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.42:44428 10.134.0.42:443 10.133.0.18:38290 - v1-completions-path 2026-11-01T10:27:02.586973Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-01T10:27:02.472Z] "DELETE /v1/api-keys/56e599f4-d227-4cf1-9017-2715e289d4e7 HTTP/1.1" 200 - via_upstream - "-" 0 365 114 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "bedf7216-ebdb-4f65-8a1a-0504942a7a64" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.133.0.149:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.42:52394 10.134.0.42:443 10.133.0.18:38298 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T10:27:16.244455Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:27:26.847479Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:27:31.929484Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:27:38.125018Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:27:45.690346Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-01T10:27:45.563Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 126 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "8ae3fa4c-52a0-4473-9443-72e33500c177" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.1.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.42:49718 10.134.0.42:443 10.133.0.18:33932 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T10:27:45.875Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 69 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "165da6b3-874e-4fa3-ae81-409bbd0cde8c" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.42:44424 10.134.0.42:443 10.133.0.18:33944 - v1-completions-path 2026-11-01T10:27:49.497165Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:28:12.931304Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:28:23.699924Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:28:28.666167Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:28:46.646327Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:28:52.918149Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:29:16.925165Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:29:36.956685Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:29:47.890730Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:29:58.832983Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:30:09.834929Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:30:20.575057Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:30:31.541824Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:30:42.318359Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:30:53.063194Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:31:03.882204Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T10:31:14.504844Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T10:31:20.544Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 281 133 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "4e81cf7b-29da-4d11-ba5b-b2404362ba2d" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.42:48914 10.134.0.42:443 10.133.0.18:54570 - v1-completions-path [2026-11-01T10:31:21.177Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 283 72 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "4a6ccf44-f18e-4ca4-9ab3-c9416cec75b9" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.42:46628 10.134.0.42:443 10.133.0.18:54574 - v1-completions-path