2026-11-01T15:56:50.094122Z info FLAG: --concurrency="0" 2026-11-01T15:56:50.094143Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-01T15:56:50.094147Z info FLAG: --help="false" 2026-11-01T15:56:50.094149Z info FLAG: --log_as_json="false" 2026-11-01T15:56:50.094151Z info FLAG: --log_caller="" 2026-11-01T15:56:50.094153Z info FLAG: --log_output_level="default:info" 2026-11-01T15:56:50.094155Z info FLAG: --log_stacktrace_level="default:none" 2026-11-01T15:56:50.094160Z info FLAG: --log_target="[stdout]" 2026-11-01T15:56:50.094162Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-01T15:56:50.094164Z info FLAG: --outlierLogPath="" 2026-11-01T15:56:50.094166Z info FLAG: --profiling="true" 2026-11-01T15:56:50.094168Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-01T15:56:50.094170Z info FLAG: --proxyLogLevel="warning" 2026-11-01T15:56:50.094172Z info FLAG: --serviceCluster="istio-proxy" 2026-11-01T15:56:50.094174Z info FLAG: --stsPort="0" 2026-11-01T15:56:50.094175Z info FLAG: --templateFile="" 2026-11-01T15:56:50.094177Z info FLAG: --tokenManagerPlugin="" 2026-11-01T15:56:50.094180Z info FLAG: --vklog="0" 2026-11-01T15:56:50.094182Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-01T15:56:50.094326Z info Proxy role ips=[10.134.0.32] type=router id=maas-default-gateway-openshift-default-7f5667f86c-8s26k.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-01T15:56:50.094374Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-01T15:56:50.095426Z info cpu limit detected as 2, setting concurrency 2026-11-01T15:56:50.095657Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-01T15:56:50.095668Z info JWT policy is third-party-jwt 2026-11-01T15:56:50.095671Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-01T15:56:50.098289Z info Starting default Istio SDS Server 2026-11-01T15:56:50.098317Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-01T15:56:50.098349Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-01T15:56:50.099057Z info Opening status port 15020 2026-11-01T15:56:50.099813Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-01T15:56:50.100478Z info sds Starting SDS grpc server 2026-11-01T15:56:50.100500Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-01T15:56:50.101301Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-01T15:56:50.102265Z info Starting proxy agent 2026-11-01T15:56:50.102295Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-01T15:56:50.159204Z info cache generated new workload certificate resourceName=default latency=58.507358ms ttl=23h59m59.840799437s 2026-11-01T15:56:50.159241Z info cache Root cert has changed, start rotating root cert 2026-11-01T15:56:50.159276Z info cache returned workload trust anchor from cache ttl=23h59m59.840725126s 2026-11-01T15:56:50.163625Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-01T15:56:50.257488Z info ads ADS: new connection for node:1 2026-11-01T15:56:50.257546Z info cache returned workload certificate from cache ttl=23h59m59.742455296s 2026-11-01T15:56:50.258006Z info ads ADS: new connection for node:2 2026-11-01T15:56:50.258107Z info cache returned workload trust anchor from cache ttl=23h59m59.741894322s 2026-11-01T15:56:50.338698Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T15:56:58.218125Z info Readiness succeeded in 8.126444491s 2026-11-01T15:56:58.218524Z info Envoy proxy is ready 2026-11-01T15:57:13.215003Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T15:57:22.956580Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T15:57:23.364266Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T15:57:58.577Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 61 - "52.71.36.72,10.132.0.17" "curl/7.76.1" "820406ef-1efe-4c5d-bea9-72dcc4ee638d" "maas.apps.c9dd05f644fcbaf91ebe.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.32:443 10.132.0.17:34390 - v1-model-routing 2026-11-01T15:57:59.526906Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T15:57:59.526953Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T15:57:59.526975Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-01T15:57:59.526981Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-01T15:57:59.458Z] "GET /v1/subscriptions HTTP/1.1" 200 - via_upstream - "-" 0 488 68 - "52.71.36.72,10.132.0.17" "curl/7.76.1" "720a20c1-d403-402e-9445-1baa2f70b013" "maas.apps.c9dd05f644fcbaf91ebe.prod.konflux-ocp-ci.dev" "10.134.0.26:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.32:43702 10.134.0.32:443 10.132.0.17:34394 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T16:00:07.951256Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T16:00:07.951302Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T16:00:07.951322Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T16:00:07.951330Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-01T16:00:07.559Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 44 - "52.71.36.72,10.132.0.17" "python-requests/2.34.2" "7f12638b-d684-42bb-bb61-daa47b8ffda8" "maas.apps.c9dd05f644fcbaf91ebe.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.134.0.32:443 10.132.0.17:59320 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T16:00:07.883Z] "DELETE /v1/api-keys/04333b25-4ad2-4d5e-8ede-7918c53df497 HTTP/1.1" 200 - via_upstream - "-" 0 321 68 - "52.71.36.72,10.132.0.17" "python-requests/2.34.2" "d6904d4a-000d-4356-b623-ac20bf12c7be" "maas.apps.c9dd05f644fcbaf91ebe.prod.konflux-ocp-ci.dev" "10.134.0.26:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.32:36138 10.134.0.32:443 10.132.0.17:59330 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T16:00:09.577909Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T16:00:09.578065Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T16:00:09.578146Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T16:00:09.578205Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-01T16:00:09.474Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 317 103 - "52.71.36.72,10.132.0.17" "python-requests/2.34.2" "cc07d50c-337e-4f2c-96d3-24fb9548511a" "maas.apps.c9dd05f644fcbaf91ebe.prod.konflux-ocp-ci.dev" "10.134.0.26:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.32:36142 10.134.0.32:443 10.132.0.17:59336 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T16:00:14.022405Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T16:00:14.022441Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T16:00:14.022454Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T16:00:14.022459Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-01T16:00:13.956Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 65 - "52.71.36.72,10.132.0.17" "python-requests/2.34.2" "85ecc1f5-b3d8-45f8-bf84-5e4e84b572e3" "maas.apps.c9dd05f644fcbaf91ebe.prod.konflux-ocp-ci.dev" "10.134.0.26:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.32:36138 10.134.0.32:443 10.132.0.17:59350 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T16:00:25.044913Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:00:40.950598Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:03:55.677480Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T16:03:55.677629Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T16:03:55.677679Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T16:03:55.677709Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-01T16:03:55.066Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 321 611 - "52.71.36.72,10.132.0.17" "python-requests/2.34.2" "769669d3-6355-4bbf-bb9c-7e2fdcfca006" "maas.apps.c9dd05f644fcbaf91ebe.prod.konflux-ocp-ci.dev" "10.134.0.26:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.32:38118 10.134.0.32:443 10.132.0.17:34766 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T16:04:13.006746Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:04:23.559178Z warn xdsproxy sendDownstreamDelta took 10.523372483s 2026-11-01T16:07:02.942615Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:07:48.518744Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:09:18.030322Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:10:59.281462Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:28:31.450914Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-01T16:34:10.354306Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T16:34:10.354341Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T16:34:10.354356Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-01T16:34:10.354361Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-01T16:34:10.271Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 69 312 82 - "52.71.36.72,10.132.0.17" "python-requests/2.34.2" "24897233-5fb1-4fbd-8d3e-e89bbc0474f1" "maas.apps.c9dd05f644fcbaf91ebe.prod.konflux-ocp-ci.dev" "10.134.0.26:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.32:44178 10.134.0.32:443 10.132.0.17:42790 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T16:34:10.571Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 110 321 138 - "52.71.36.72,10.132.0.17" "python-requests/2.34.2" "5e6c4881-a3b7-4f55-9783-56d9abee291c" "maas.apps.c9dd05f644fcbaf91ebe.prod.konflux-ocp-ci.dev" "10.134.0.30:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.32:60746 10.134.0.32:443 10.132.0.17:42806 - v1-chat-completions-path 2026-11-01T16:34:13.644497Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T16:34:13.644540Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T16:34:13.644558Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T16:34:13.644564Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T16:34:13.644766Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 2026-11-01T16:34:13.644828Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-01T16:34:13.644839Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-01T16:34:13.644843Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-01T16:34:13.644847Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-11-01T16:34:13.583Z] "POST /v1/chat/completions HTTP/1.1" 404 - via_upstream - "-" 120 126 61 - "52.71.36.72,10.132.0.17" "python-requests/2.34.2" "9310d342-35cc-48e3-8a17-b7a69ac1dce3" "maas.apps.c9dd05f644fcbaf91ebe.prod.konflux-ocp-ci.dev" "10.134.0.30:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.32:60762 10.134.0.32:443 10.132.0.17:42820 - v1-chat-completions-path 2026-11-01T16:36:06.540702Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:36:28.906394Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:38:12.990090Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:38:16.043769Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-01T16:38:16.037Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 74 - "52.71.36.72,10.132.0.17" "python-requests/2.34.2" "10338df1-a6b8-4df9-b84b-d6ab759d8959" "maas.apps.c9dd05f644fcbaf91ebe.prod.konflux-ocp-ci.dev" "10.134.0.28:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.32:48476 10.134.0.32:443 10.132.0.17:50522 - v1-completions-path 2026-11-01T16:38:28.421619Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:38:30.019499Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T16:38:35.114Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 64 - "52.71.36.72,10.132.0.17" "python-requests/2.34.2" "5c38f364-d148-4017-9524-a28df7823376" "maas.apps.c9dd05f644fcbaf91ebe.prod.konflux-ocp-ci.dev" "10.134.0.28:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.32:48476 10.134.0.32:443 10.132.0.17:55550 - v1-completions-path 2026-11-01T16:38:47.102471Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:38:49.041671Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:38:52.300092Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:38:59.771441Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:39:03.007841Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=3 2026-11-01T16:39:06.333586Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-11-01T16:39:06.279Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 59 - "52.71.36.72,10.132.0.17" "python-requests/2.34.2" "c9ee1dac-9ff3-46e0-85db-b2dfc9939b2b" "maas.apps.c9dd05f644fcbaf91ebe.prod.konflux-ocp-ci.dev" "10.134.0.28:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.32:34734 10.134.0.32:443 10.132.0.17:59950 - v1-completions-path [2026-11-01T16:39:09.033Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 57 - "52.71.36.72,10.132.0.17" "python-requests/2.34.2" "d2f10412-abd9-40f8-9e2b-ab6801405901" "maas.apps.c9dd05f644fcbaf91ebe.prod.konflux-ocp-ci.dev" "10.134.0.28:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.32:48476 10.134.0.32:443 10.132.0.17:55366 - v1-completions-path 2026-11-01T16:39:24.374776Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:39:34.562481Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:39:36.250169Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:39:55.379948Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:39:57.097329Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:40:13.589045Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:40:15.854641Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:40:32.484903Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:40:34.064913Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:40:50.126141Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:41:00.705890Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:41:03.634807Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:41:18.745646Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:41:20.243148Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:41:29.032815Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:41:47.457660Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:41:57.576867Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:42:13.059916Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:42:15.683469Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:43:52.798769Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:43:55.773996Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:45:21.551302Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T16:45:24.565Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 73 302 80 - "52.71.36.72,10.132.0.17" "python-requests/2.34.2" "69a95956-65c6-4ee0-a472-35d83f2cafc7" "maas.apps.c9dd05f644fcbaf91ebe.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-trlp-test-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.32:53266 10.134.0.32:443 10.132.0.17:56732 - v1-completions-path 2026-11-01T16:45:37.014054Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:45:38.832689Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:45:55.953159Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:47:22.550147Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:47:38.802832Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:47:54.431381Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:48:06.170403Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:48:10.062463Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:48:17.757910Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:48:20.278384Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:48:25.881064Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T16:48:25.881109Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T16:48:25.881132Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T16:48:25.881139Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-01T16:48:25.819Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 335 61 - "52.71.36.72,10.132.0.17" "python-requests/2.34.2" "2debe90b-807c-4815-ad79-b52115c0aee5" "maas.apps.c9dd05f644fcbaf91ebe.prod.konflux-ocp-ci.dev" "10.134.0.26:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.32:40938 10.134.0.32:443 10.132.0.17:60062 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T16:48:38.079191Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:51:19.167231Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-01T16:51:19.167260Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-01T16:51:19.167265Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-01T16:51:19.167270Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-11-01T16:51:19.111Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 403 - - "-" 61 103 56 - "52.71.36.72,10.132.0.17" "python-requests/2.34.2" "f2dbec1f-6172-466b-9cf6-0dacedfd0d06" "maas.apps.c9dd05f644fcbaf91ebe.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.32:443 10.132.0.17:45916 - v1-catch-all-path 2026-11-01T16:51:21.218130Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:51:26.254130Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:51:41.661588Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:51:43.570195Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:51:58.274997Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T16:52:09.170793Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8