2026-10-30T20:14:12.817935Z info FLAG: --concurrency="0" 2026-10-30T20:14:12.817954Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T20:14:12.817967Z info FLAG: --help="false" 2026-10-30T20:14:12.817969Z info FLAG: --log_as_json="false" 2026-10-30T20:14:12.817971Z info FLAG: --log_caller="" 2026-10-30T20:14:12.817973Z info FLAG: --log_output_level="default:info" 2026-10-30T20:14:12.817975Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T20:14:12.817980Z info FLAG: --log_target="[stdout]" 2026-10-30T20:14:12.817982Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T20:14:12.817984Z info FLAG: --outlierLogPath="" 2026-10-30T20:14:12.817988Z info FLAG: --profiling="true" 2026-10-30T20:14:12.817990Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T20:14:12.817992Z info FLAG: --proxyLogLevel="warning" 2026-10-30T20:14:12.817994Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T20:14:12.817996Z info FLAG: --stsPort="0" 2026-10-30T20:14:12.817998Z info FLAG: --templateFile="" 2026-10-30T20:14:12.818000Z info FLAG: --tokenManagerPlugin="" 2026-10-30T20:14:12.818003Z info FLAG: --vklog="0" 2026-10-30T20:14:12.818005Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T20:14:12.818155Z info Proxy role ips=[10.132.0.45] type=router id=maas-default-gateway-openshift-default-64d4cc998c-pzpq5.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T20:14:12.818239Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T20:14:12.819910Z info cpu limit detected as 2, setting concurrency 2026-10-30T20:14:12.820114Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T20:14:12.820125Z info JWT policy is third-party-jwt 2026-10-30T20:14:12.820128Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T20:14:12.821548Z info Starting default Istio SDS Server 2026-10-30T20:14:12.821573Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T20:14:12.821598Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T20:14:12.822108Z info Opening status port 15020 2026-10-30T20:14:12.823066Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T20:14:12.823772Z info sds Starting SDS grpc server 2026-10-30T20:14:12.823912Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T20:14:12.824730Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T20:14:12.825807Z info Starting proxy agent 2026-10-30T20:14:12.825838Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T20:14:12.891182Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T20:14:12.899719Z info cache generated new workload certificate resourceName=default latency=75.696602ms ttl=23h59m59.100284112s 2026-10-30T20:14:12.899758Z info cache Root cert has changed, start rotating root cert 2026-10-30T20:14:12.899799Z info cache returned workload trust anchor from cache ttl=23h59m59.100202066s 2026-10-30T20:14:12.979125Z info ads ADS: new connection for node:1 2026-10-30T20:14:12.979197Z info cache returned workload certificate from cache ttl=23h59m59.020805354s 2026-10-30T20:14:12.979464Z info ads ADS: new connection for node:2 2026-10-30T20:14:12.979537Z info cache returned workload trust anchor from cache ttl=23h59m59.020464517s 2026-10-30T20:14:13.024492Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:14:18.748652Z info Readiness succeeded in 5.933469706s 2026-10-30T20:14:18.748999Z info Envoy proxy is ready 2026-10-30T20:14:25.008159Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:15:02.413665Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:15:04.058898Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:15:05.541707Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:15:07.633023Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:15:09.749760Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:15:11.394085Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:15:13.221816Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:15:31.521067Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T20:16:07.712Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 141 - "52.71.36.72,10.132.0.11" "curl/7.76.1" "1d47d25f-f23e-4d69-8aff-ac0ee2c3f947" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.45:443 10.132.0.11:43554 - v1-model-routing 2026-10-30T20:16:08.924059Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:16:08.924097Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:16:08.924111Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T20:16:08.924116Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:16:09.003823Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:16:09.003865Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:16:09.003882Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:16:09.003888Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T20:16:09.133632Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:16:09.133664Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:16:09.133679Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T20:16:09.133684Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T20:16:08.424Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 115 - "52.71.36.72,10.132.0.11" "curl/7.76.1" "4eb81bb6-e61a-4f27-b0f0-86313292c43b" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.45:443 10.132.0.11:43576 - v1-model-routing [2026-10-30T20:16:08.853Z] "DELETE /v1/api-keys/ef5869a9-928e-4f75-a69f-a57e39e55f35 HTTP/1.1" 200 - via_upstream - "-" 0 365 70 - "52.71.36.72,10.132.0.11" "curl/7.76.1" "201b5010-78c7-4f00-bbe5-c8eb17bb8c24" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "10.132.0.49:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.45:46148 10.132.0.45:443 10.132.0.11:43580 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T20:16:08.995Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 20 8 - "52.71.36.72,10.132.0.11" "curl/7.76.1" "82b3c2cb-2d7c-47de-87dc-db92a82a1975" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "10.132.0.49:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.45:46164 10.132.0.45:443 10.132.0.11:43588 - odh-ai-gateway-infra.maas-api-route.4 [2026-10-30T20:16:09.070Z] "GET /v1/subscriptions HTTP/1.1" 200 - via_upstream - "-" 0 488 63 - "52.71.36.72,10.132.0.11" "curl/7.76.1" "b0ccf2af-87f7-4036-8da2-6c5868164706" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "10.132.0.49:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.45:46148 10.132.0.45:443 10.132.0.11:43596 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T20:18:03.627252Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:18:03.627284Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:18:03.627298Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T20:18:03.627302Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T20:18:03.552Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 84 326 75 - "52.71.36.72,10.132.0.11" "python-requests/2.34.2" "11a0547d-7a20-91b7-b9dc-084d9142dd9f" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "10.132.0.49:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.45:36560 10.132.0.45:443 10.132.0.11:35304 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T20:18:04.732862Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:18:04.732894Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:18:04.732908Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T20:18:04.732912Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:18:04.863849Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:18:04.863881Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:18:04.863895Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T20:18:04.863899Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T20:18:05.260240Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T20:18:05.260263Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T20:18:05.260266Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T20:18:05.260270Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-10-30T20:18:04.614Z] "DELETE /v1/api-keys/56a3c69d-39c0-4df9-a5ed-ad8af1bed948 HTTP/1.1" 200 - via_upstream - "-" 0 321 118 - "52.71.36.72,10.132.0.11" "python-requests/2.34.2" "c2c90ffb-ce78-4d5f-bdca-37ccca902d73" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "10.132.0.49:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.45:36560 10.132.0.45:443 10.132.0.11:35314 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T20:18:04.805Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 58 - "52.71.36.72,10.132.0.11" "python-requests/2.34.2" "e107cd8a-b5b8-4035-a5a1-2b560814b457" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "10.132.0.49:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.45:36576 10.132.0.45:443 10.132.0.11:35328 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T20:18:05.123Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 77 137 - "52.71.36.72,10.132.0.11" "python-requests/2.34.2" "ff2ece99-0605-4596-9eb3-b6bba0f40393" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.45:443 10.132.0.11:35332 - v1-completions-path 2026-10-30T20:18:05.638149Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:18:05.638182Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:18:05.638196Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T20:18:05.638201Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T20:18:05.735423Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T20:18:05.735444Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T20:18:05.735447Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T20:18:05.735450Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-10-30T20:18:05.509Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 128 - "52.71.36.72,10.132.0.11" "python-requests/2.34.2" "b9fd25b5-9dd0-46bc-9569-f5aa52ac2b71" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "10.132.0.49:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.45:36560 10.132.0.45:443 10.132.0.11:35338 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T20:18:05.698Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 76 77 37 - "52.71.36.72,10.132.0.11" "python-requests/2.34.2" "4d4c2d76-a285-4969-8fc5-5e3409d4de33" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.45:443 10.132.0.11:35344 - v1-completions-path [2026-10-30T20:18:11.574Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 281 111 - "52.71.36.72,10.132.0.11" "python-requests/2.34.2" "1741b1a7-d579-4e90-93bd-8f65324f3eff" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:46510 10.132.0.45:443 10.132.0.11:48012 - v1-completions-path 2026-10-30T20:18:13.807884Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-10-30T20:18:13.723Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 288 87 - "52.71.36.72,10.132.0.11" "-" "9832939f-b357-483c-89ed-86d373d8b9a9" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:46510 10.132.0.45:443 10.132.0.11:48022 - v1-completions-path 2026-10-30T20:18:20.017612Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:18:33.535626Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:21:51.801019Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: gRPC status code is not OK thread=24 [2026-10-30T20:21:49.784Z] "POST /maas-api/v1/tokens HTTP/1.1" 500 - - "-" 20 23 2017 - "52.71.36.72,10.132.0.11" "python-requests/2.34.2" "652efae1-0bdc-41b1-9825-064e5c60b0fa" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.132.0.45:443 10.132.0.11:44072 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T20:22:04.967Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 136 337 77 - "52.71.36.72,10.132.0.11" "python-requests/2.34.2" "1489a955-49a1-4703-a2cc-de22dae538fa" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:46510 10.132.0.45:443 10.132.0.11:45692 - v1-chat-completions-path 2026-10-30T20:22:25.172664Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-10-30T20:22:25.267416Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T20:22:25.267739Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-10-30T20:22:25.024Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 135 2330 243 - "52.71.36.72,10.132.0.11" "python-requests/2.34.2" "5713fa84-9ebc-46c2-ab32-c5376c4b4613" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:37236 10.132.0.45:443 10.132.0.11:36444 - v1-chat-completions-path 2026-10-30T20:24:28.529239Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:24:52.585458Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:25:31.383647Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:25:46.846546Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:25:48.446648Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:26:05.792806Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:26:07.452047Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:26:10.472665Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:26:18.429656Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T20:26:23.131Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 70 - "52.71.36.72,10.132.0.11" "python-requests/2.34.2" "9e431d35-f146-4c74-a5a5-28f55298f9f7" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "10.132.0.52:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:35484 10.132.0.45:443 10.132.0.11:45864 - v1-completions-path [2026-10-30T20:26:23.849Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 56 - "52.71.36.72,10.132.0.11" "python-requests/2.34.2" "93eb1619-72cf-4e0a-96b9-234ec543cc63" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "10.132.0.52:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:35484 10.132.0.45:443 10.132.0.11:45872 - v1-completions-path 2026-10-30T20:26:39.594021Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:26:50.011248Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:26:52.039216Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:27:10.787429Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:27:12.420881Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T20:27:17.602Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 74 289 77 - "52.71.36.72,10.132.0.11" "python-requests/2.34.2" "2d48273b-cdf6-461a-bd10-9316d1e3e1c1" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "10.133.0.30:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:46800 10.132.0.45:443 10.132.0.11:38178 - v1-completions-path 2026-10-30T20:27:29.174155Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:27:33.380646Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:27:47.317381Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:27:48.852509Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T20:28:04.533Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 281 60 - "52.71.36.72,10.132.0.11" "python-requests/2.34.2" "1b26c316-d5d1-40ec-85f2-0679aa5b44f4" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:46510 10.132.0.45:443 10.132.0.11:53810 - v1-completions-path 2026-10-30T20:28:05.381827Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:28:16.106856Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:28:19.184052Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:28:34.721651Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:28:36.692254Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:28:54.821603Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:28:56.334643Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:30:25.613650Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:30:28.331821Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:31:52.348131Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:32:07.276874Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:32:08.800093Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:32:23.307790Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:33:39.614482Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:33:54.251786Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:34:10.733463Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:34:22.495246Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:34:26.340650Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:34:37.215085Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:34:39.490153Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:34:58.810944Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:35:06.708509Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T20:35:06.708533Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T20:35:06.708536Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T20:35:06.708540Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-10-30T20:35:06.664Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 403 - - "-" 61 103 43 - "52.71.36.72,10.132.0.11" "python-requests/2.34.2" "dd0811e2-2600-4f81-8eee-30c9f0df057e" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.45:443 10.132.0.11:35700 - v1-catch-all-path 2026-10-30T20:35:08.442434Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:35:11.810795Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-10-30T20:35:11.742Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 91 3616 68 - "52.71.36.72,10.132.0.11" "python-requests/2.34.2" "d7610d8a-df7f-4433-bfa4-25e10b93983c" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "10.132.0.51:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:46270 10.132.0.45:443 10.132.0.11:37414 - v1-catch-all-model-routing 2026-10-30T20:35:13.591776Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:35:16.551969Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-10-30T20:35:16.487Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 64 - "52.71.36.72,10.132.0.11" "python-requests/2.34.2" "697bf5ae-cca4-461e-a026-8c7c54672167" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "10.132.0.51:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:46280 10.132.0.45:443 10.132.0.11:37432 - v1-catch-all-path [2026-10-30T20:35:17.161Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 429 - - "-" 61 18 77 - "52.71.36.72,10.132.0.11" "python-requests/2.34.2" "a25fa14a-3611-4e30-a919-71031a91b121" "maas.apps.3d1a0c34cf950dbd372c.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.45:443 10.132.0.11:37446 - v1-catch-all-path 2026-10-30T20:35:28.421818Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:35:29.992982Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:35:44.239109Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T20:35:55.327086Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8