2026-11-01T00:28:56.965204Z info FLAG: --concurrency="0" 2026-11-01T00:28:56.965223Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-01T00:28:56.965227Z info FLAG: --help="false" 2026-11-01T00:28:56.965229Z info FLAG: --log_as_json="false" 2026-11-01T00:28:56.965231Z info FLAG: --log_caller="" 2026-11-01T00:28:56.965233Z info FLAG: --log_output_level="default:info" 2026-11-01T00:28:56.965235Z info FLAG: --log_stacktrace_level="default:none" 2026-11-01T00:28:56.965240Z info FLAG: --log_target="[stdout]" 2026-11-01T00:28:56.965242Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-01T00:28:56.965244Z info FLAG: --outlierLogPath="" 2026-11-01T00:28:56.965246Z info FLAG: --profiling="true" 2026-11-01T00:28:56.965248Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-01T00:28:56.965250Z info FLAG: --proxyLogLevel="warning" 2026-11-01T00:28:56.965252Z info FLAG: --serviceCluster="istio-proxy" 2026-11-01T00:28:56.965254Z info FLAG: --stsPort="0" 2026-11-01T00:28:56.965255Z info FLAG: --templateFile="" 2026-11-01T00:28:56.965257Z info FLAG: --tokenManagerPlugin="" 2026-11-01T00:28:56.965260Z info FLAG: --vklog="0" 2026-11-01T00:28:56.965263Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-01T00:28:56.965393Z info Proxy role ips=[10.132.1.70] type=router id=maas-default-gateway-openshift-default-6cc458d887-795lj.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-01T00:28:56.965444Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-01T00:28:56.966553Z info cpu limit detected as 2, setting concurrency 2026-11-01T00:28:56.966753Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-01T00:28:56.966764Z info JWT policy is third-party-jwt 2026-11-01T00:28:56.966782Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-01T00:28:56.968028Z info Starting default Istio SDS Server 2026-11-01T00:28:56.968049Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-01T00:28:56.968076Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-01T00:28:56.968887Z info Opening status port 15020 2026-11-01T00:28:56.969403Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-01T00:28:56.970691Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-01T00:28:56.971530Z info Starting proxy agent 2026-11-01T00:28:56.971560Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-01T00:28:56.981150Z info sds Starting SDS grpc server 2026-11-01T00:28:56.981170Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-01T00:28:57.037233Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-01T00:28:57.135678Z info cache generated new workload certificate resourceName=default latency=166.562177ms ttl=23h59m59.864326095s 2026-11-01T00:28:57.135712Z info cache Root cert has changed, start rotating root cert 2026-11-01T00:28:57.135744Z info cache returned workload trust anchor from cache ttl=23h59m59.864256858s 2026-11-01T00:28:57.145939Z info ads ADS: new connection for node:1 2026-11-01T00:28:57.145998Z info cache returned workload certificate from cache ttl=23h59m59.854003262s 2026-11-01T00:28:57.146192Z info ads ADS: new connection for node:2 2026-11-01T00:28:57.146265Z info cache returned workload trust anchor from cache ttl=23h59m59.853735597s 2026-11-01T00:28:57.236050Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:29:05.101048Z info Readiness succeeded in 8.138906997s 2026-11-01T00:29:05.101507Z info Envoy proxy is ready 2026-11-01T00:29:24.016989Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:29:39.785930Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:29:42.212953Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:29:48.612496Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:29:55.727367Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:30:05.614024Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:30:20.400211Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:30:31.428728Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:30:33.219567Z info ads ADS: new connection for node:3 2026-11-01T00:30:33.219757Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-01T00:30:33.219828Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-01T00:30:34.525930Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:31:15.936220Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T00:31:33.835Z] "POST /v1/chat/completions HTTP/1.1" 403 - - "-" 131 0 44 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "e8af72a2-7c4e-47c1-b6da-937cb5fe38ca" "maas.apps.84f85b540f82405c5762.prod.konflux-ocp-ci.dev" "-" outbound|54321||e2e-prio-pool-ca03fe61-inference-pool-ip-13923081.llm.svc.cluster.local - 10.132.1.70:443 10.132.0.9:53682 - v1-model-routing 2026-11-01T00:31:35.298421Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:31:35.319740Z info ads ADS: "" 3 terminated with stream closed 2026-11-01T00:31:37.700027Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:31:40.523400Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:31:45.614272Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:34:05.294676Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:34:21.854946Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:34:38.329264Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:34:50.271694Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:34:54.126189Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:35:05.877649Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:35:09.415836Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:35:25.110257Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:35:27.856512Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T00:35:27.856549Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T00:35:27.856565Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T00:35:27.856570Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-01T00:35:27.775Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 88 330 81 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "efe13ab0-5f86-407e-a794-bce66d53aec3" "maas.apps.84f85b540f82405c5762.prod.konflux-ocp-ci.dev" "10.133.0.35:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.70:47728 10.132.1.70:443 10.132.0.9:42042 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T00:36:12.262547Z info ads ADS: new connection for node:4 2026-11-01T00:36:12.266662Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-01T00:36:12.266715Z info cache read certificate from file resource=file-root:system 2026-11-01T00:36:13.419456Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:36:14.993814Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:36:43.307598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T00:36:43.307626Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T00:36:43.307634Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T00:36:43.307639Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-01T00:36:43.307813Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-01T00:36:43.189Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1707 117 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "81e8344b-6cdd-43b6-8018-4a4db7676c78" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.1.70:35584 10.132.1.70:443 10.132.0.9:33672 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-01T00:36:53.972968Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T00:36:53.973000Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T00:36:53.973010Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T00:36:53.973015Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T00:36:53.973183Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-01T00:36:53.860Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1717 113 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "7152fb36-b0ce-46e1-84b7-0805147f3fe3" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.1.70:54952 10.132.1.70:443 10.132.0.9:48886 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-01T00:36:56.099968Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T00:36:56.099999Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T00:36:56.100009Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T00:36:56.100013Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T00:36:56.100159Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-01T00:36:55.996Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1713 103 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "983321f8-a1d0-4adb-908d-168eb4d08df7" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.1.70:54952 10.132.1.70:443 10.132.0.9:48898 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-01T00:37:17.027846Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T00:37:17.027872Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T00:37:17.027881Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T00:37:17.027885Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T00:37:17.028046Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-01T00:37:13.899Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1713 3128 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "61cff5f5-92ed-494f-a6a2-181d22b1eccf" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.1.70:54952 10.132.1.70:443 10.132.0.9:46760 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-01T00:37:38.213525Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-01T00:37:38.252258Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T00:37:38.252284Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T00:37:38.252293Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T00:37:38.252297Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T00:37:38.252453Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-01T00:37:38.155Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1713 96 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "7d012e1b-3578-4328-8d48-0e13db6b28cb" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.1.70:54952 10.132.1.70:443 10.132.0.9:55422 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-01T00:37:43.394175Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T00:37:43.394209Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T00:37:43.394221Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-01T00:37:43.394226Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T00:37:43.394430Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-01T00:37:40.276Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1713 3118 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "4c049fbb-fcc4-4f00-9132-fe9cdd5812dd" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.1.70:35584 10.132.1.70:443 10.132.0.9:55428 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-01T00:37:56.803705Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:37:56.952073Z info ads ADS: "" 4 terminated 2026-11-01T00:37:58.288968Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:38:05.087367Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:38:20.703660Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:38:25.295513Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:38:28.252220Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T00:38:28.252254Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T00:38:28.252268Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T00:38:28.252273Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-01T00:38:28.174Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 335 78 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "b224d3dd-ad8d-4275-b15a-f900f944a12e" "maas.apps.84f85b540f82405c5762.prod.konflux-ocp-ci.dev" "10.133.0.35:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.70:38776 10.132.1.70:443 10.132.0.9:52036 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T00:38:40.523800Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:38:56.573900Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:39:01.977456Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-01T00:39:01.925Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 72 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "6a12ea4b-be1e-43ae-8816-52f4e955e137" "maas.apps.84f85b540f82405c5762.prod.konflux-ocp-ci.dev" "10.134.0.46:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.70:57732 10.132.1.70:443 10.132.0.9:60720 - v1-completions-path [2026-11-01T00:39:02.520Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 58 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "926240cd-2f2d-4ac7-9fc7-bc2f4c5dbc7d" "maas.apps.84f85b540f82405c5762.prod.konflux-ocp-ci.dev" "10.134.0.46:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.70:57732 10.132.1.70:443 10.132.0.9:60722 - v1-completions-path [2026-11-01T00:39:03.402Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 56 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "d87bd54e-fe28-41bd-b08c-83eef5cb32c6" "maas.apps.84f85b540f82405c5762.prod.konflux-ocp-ci.dev" "10.134.0.46:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.70:57732 10.132.1.70:443 10.132.0.9:60732 - v1-completions-path [2026-11-01T00:39:03.477Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 55 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "2591df36-a3af-4818-b8e0-5e190880d76f" "maas.apps.84f85b540f82405c5762.prod.konflux-ocp-ci.dev" "10.134.0.46:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.70:57740 10.132.1.70:443 10.132.0.9:60746 - v1-completions-path 2026-11-01T00:39:20.042716Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:39:30.313656Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:39:34.887906Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:39:51.832135Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:39:54.175839Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:40:01.269123Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:40:13.192497Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:40:18.417707Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:40:26.730263Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:40:31.994865Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:40:50.288897Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:40:53.271418Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:40:56.429733Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T00:40:56.429783Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T00:40:56.429805Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-01T00:40:56.429810Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-01T00:40:56.351Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 324 78 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "d7a674e8-74cb-48f0-bbb4-0d5494577c39" "maas.apps.84f85b540f82405c5762.prod.konflux-ocp-ci.dev" "10.133.0.35:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.70:55684 10.132.1.70:443 10.132.0.9:59090 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T00:41:10.413652Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T00:41:10.413688Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T00:41:10.413704Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T00:41:10.413710Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-01T00:41:10.339Z] "DELETE /v1/api-keys/09026a91-5a1c-4fee-aaec-a6655709b1bc HTTP/1.1" 200 - via_upstream - "-" 0 375 74 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "28012d54-0294-45c8-ba21-4b3f401575cd" "maas.apps.84f85b540f82405c5762.prod.konflux-ocp-ci.dev" "10.133.0.35:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.70:59898 10.132.1.70:443 10.132.0.9:42656 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T00:41:17.124297Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:41:35.113882Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:41:45.360153Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T00:43:56.318Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 47 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "8011fbda-b644-4a6d-b453-467675bba990" "maas.apps.84f85b540f82405c5762.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.1.70:443 10.132.0.9:39034 - v1-completions-path 2026-11-01T00:45:26.182688Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:45:41.568410Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:45:43.860909Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:45:50.984486Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:46:10.764684Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:46:14.383959Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T00:46:17.490Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 59 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "da7aa355-4a3e-4254-896f-6920f2803686" "maas.apps.84f85b540f82405c5762.prod.konflux-ocp-ci.dev" "10.134.0.46:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.70:57740 10.132.1.70:443 10.132.0.9:35802 - v1-completions-path 2026-11-01T00:46:29.991008Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:46:40.361269Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:46:45.463355Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:47:02.457593Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:47:06.895383Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:47:10.446190Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T00:47:16.678Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 62 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "b19cf985-ccde-451d-82ad-2b70e32b31cd" "maas.apps.84f85b540f82405c5762.prod.konflux-ocp-ci.dev" "10.134.0.46:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.70:57740 10.132.1.70:443 10.132.0.9:38162 - v1-completions-path 2026-11-01T00:47:33.137926Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:47:43.468296Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:47:47.950143Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:47:51.385057Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:47:56.302346Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T00:48:01.682Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 64 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "79158f02-b237-4012-8afc-3aa8690d1933" "maas.apps.84f85b540f82405c5762.prod.konflux-ocp-ci.dev" "10.134.0.46:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.70:57740 10.132.1.70:443 10.132.0.9:52274 - v1-completions-path 2026-11-01T00:48:14.790851Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:48:25.102073Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:48:27.318698Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:48:36.942320Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:48:40.477373Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:48:58.375982Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:49:00.591561Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:49:19.040051Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:49:29.501874Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:49:40.003987Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:49:50.275482Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:50:00.636493Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:50:10.980014Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:50:21.376072Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:50:31.733162Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:50:41.993788Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T00:50:52.376129Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T00:50:59.634Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 43 - "52.71.36.72,10.132.0.9" "python-requests/2.34.2" "c0ae89b1-a51e-4efa-a9b0-d651a71ddc09" "maas.apps.84f85b540f82405c5762.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.1.70:443 10.132.0.9:33762 - v1-completions-path