2026-11-03T10:39:24.904764Z info FLAG: --concurrency="0" 2026-11-03T10:39:24.904783Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T10:39:24.904788Z info FLAG: --help="false" 2026-11-03T10:39:24.904790Z info FLAG: --log_as_json="false" 2026-11-03T10:39:24.904792Z info FLAG: --log_caller="" 2026-11-03T10:39:24.904794Z info FLAG: --log_output_level="default:info" 2026-11-03T10:39:24.904796Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T10:39:24.904800Z info FLAG: --log_target="[stdout]" 2026-11-03T10:39:24.904803Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T10:39:24.904805Z info FLAG: --outlierLogPath="" 2026-11-03T10:39:24.904807Z info FLAG: --profiling="true" 2026-11-03T10:39:24.904809Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T10:39:24.904811Z info FLAG: --proxyLogLevel="warning" 2026-11-03T10:39:24.904813Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T10:39:24.904815Z info FLAG: --stsPort="0" 2026-11-03T10:39:24.904816Z info FLAG: --templateFile="" 2026-11-03T10:39:24.904818Z info FLAG: --tokenManagerPlugin="" 2026-11-03T10:39:24.904821Z info FLAG: --vklog="0" 2026-11-03T10:39:24.904823Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T10:39:24.905003Z info Proxy role ips=[10.134.0.37] type=router id=maas-default-gateway-openshift-default-6cc458d887-74hrv.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T10:39:24.905052Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T10:39:24.906099Z info cpu limit detected as 2, setting concurrency 2026-11-03T10:39:24.906303Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T10:39:24.906315Z info JWT policy is third-party-jwt 2026-11-03T10:39:24.906318Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T10:39:24.907740Z info Starting default Istio SDS Server 2026-11-03T10:39:24.907766Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T10:39:24.907783Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T10:39:24.908193Z info Opening status port 15020 2026-11-03T10:39:24.909416Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T10:39:24.909676Z info sds Starting SDS grpc server 2026-11-03T10:39:24.909705Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T10:39:24.910898Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T10:39:24.911727Z info Starting proxy agent 2026-11-03T10:39:24.911758Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T10:39:24.975674Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T10:39:25.056887Z info cache generated new workload certificate resourceName=default latency=147.050477ms ttl=23h59m59.943116924s 2026-11-03T10:39:25.056950Z info cache Root cert has changed, start rotating root cert 2026-11-03T10:39:25.056983Z info cache returned workload trust anchor from cache ttl=23h59m59.943017456s 2026-11-03T10:39:25.076456Z info ads ADS: new connection for node:1 2026-11-03T10:39:25.076533Z info cache returned workload certificate from cache ttl=23h59m59.923469243s 2026-11-03T10:39:25.077169Z info ads ADS: new connection for node:2 2026-11-03T10:39:25.077259Z info cache returned workload trust anchor from cache ttl=23h59m59.922741811s 2026-11-03T10:39:25.157571Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:39:32.986593Z info Readiness succeeded in 8.084412215s 2026-11-03T10:39:32.987375Z info Envoy proxy is ready 2026-11-03T10:42:43.496328Z info ads ADS: new connection for node:3 2026-11-03T10:42:43.505020Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-03T10:42:43.505149Z info cache read certificate from file resource=file-root:system 2026-11-03T10:42:43.985507Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:42:50.632110Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:42:56.719791Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:43:09.165027Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:43:18.518761Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:43:18.518798Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:43:18.518809Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T10:43:18.518814Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:43:18.519086Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T10:43:18.302Z] "POST /anything/e2e-rt-sw-5bac6a-client HTTP/1.1" 200 - via_upstream - "-" 124 1760 216 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "d1154473-d768-4732-bb98-dae04d0316b9" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.37:49568 10.134.0.37:443 10.132.0.11:59218 - llm.external-model-e2e-rt-sw-5bac6a-5a707f712a2d.0 2026-11-03T10:43:18.842974Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:43:18.843023Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:43:18.843038Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:43:18.843044Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T10:43:18.843377Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T10:43:18.606Z] "POST /anything/e2e-rt-sw-5bac6a-client HTTP/1.1" 200 - via_upstream - "-" 124 1769 237 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "6a7629f0-7eee-4d44-ba69-f6e5ed35b884" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.37:49568 10.134.0.37:443 10.132.0.11:59222 - llm.external-model-e2e-rt-sw-5bac6a-5a707f712a2d.0 2026-11-03T10:43:19.338842Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:43:19.338888Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:43:19.338906Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T10:43:19.338934Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:43:19.339208Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 2026-11-03T10:43:19.869123Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:43:19.869153Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:43:19.869163Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T10:43:19.869167Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:43:19.869309Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T10:43:19.118Z] "POST /anything/e2e-rt-sw-5bac6a-client HTTP/1.1" 200 - via_upstream - "-" 124 1767 221 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "2e44b49e-6753-4a9a-b54b-3b80f0fb9cbe" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.37:49580 10.134.0.37:443 10.132.0.11:59232 - llm.external-model-e2e-rt-sw-5bac6a-5a707f712a2d.0 [2026-11-03T10:43:19.764Z] "POST /llm/e2e-rt-sw-5bac6a/anything/e2e-rt-sw-5bac6a-client HTTP/1.1" 503 UH no_healthy_upstream - "-" 124 19 105 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "77b4b56a-46f6-4add-8d02-35ebd0cfad38" "maas.apps.9beba4aa8ba5e1c4c2c4.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rt-sw-5bac6a.llm.svc.cluster.local - 10.134.0.37:443 10.132.0.11:52622 - llm.external-model-e2e-rt-sw-5bac6a-5a707f712a2d.2 2026-11-03T10:43:32.166421Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:43:42.839311Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:43:42.839341Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:43:42.839362Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T10:43:42.839366Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:43:42.839528Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T10:43:42.712Z] "POST /anything/e2e-rt-sw-5bac6a-client HTTP/1.1" 200 - via_upstream - "-" 124 1761 126 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "a799da09-1d04-474c-bf5b-c59f18cfd7e9" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.37:50758 10.134.0.37:443 10.132.0.11:39916 - llm.external-model-e2e-rt-sw-5bac6a-5a707f712a2d.0 2026-11-03T10:43:55.995420Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:44:02.283487Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:44:02.283520Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:44:02.283531Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T10:44:02.283535Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:44:02.283692Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T10:44:02.126Z] "POST /anything/e2e-rt-eq-29fe19-client HTTP/1.1" 200 - via_upstream - "-" 124 1765 157 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "d2a587e0-6a51-4d02-8fea-b1be04a52160" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.37:33738 10.134.0.37:443 10.132.0.11:46932 - llm.external-model-e2e-rt-eq-29fe19-b99dd53071a2.2 2026-11-03T10:44:03.271760Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-11-03T10:44:03.272148Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:44:03.272172Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:44:03.272183Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:44:03.272189Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T10:44:03.272352Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T10:44:03.156Z] "POST /anything/e2e-rt-eq-29fe19-client HTTP/1.1" 200 - via_upstream - "-" 124 1763 116 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "27452c5e-a2ce-43a4-9968-d036cfa4bfc2" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.37:42196 10.134.0.37:443 10.132.0.11:46938 - llm.external-model-e2e-rt-eq-29fe19-b99dd53071a2.0 2026-11-03T10:44:04.268192Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:44:04.268226Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:44:04.268238Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T10:44:04.268242Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:44:04.268411Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T10:44:04.146Z] "POST /anything/e2e-rt-eq-29fe19-client HTTP/1.1" 200 - via_upstream - "-" 124 1756 121 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "3159e20a-3366-4aa9-837f-6c21930f78a5" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.37:33738 10.134.0.37:443 10.132.0.11:46964 - llm.external-model-e2e-rt-eq-29fe19-b99dd53071a2.0 2026-11-03T10:44:23.166881Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:44:29.156791Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:44:29.200960Z info ads ADS: "" 3 terminated 2026-11-03T10:44:41.321277Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:47:58.083240Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:47:58.083295Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:47:58.083318Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T10:47:58.083326Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T10:47:57.841Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 84 327 242 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "aaa40adb-2225-4b4e-bfc5-767547a2630c" "maas.apps.9beba4aa8ba5e1c4c2c4.prod.konflux-ocp-ci.dev" "10.134.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.37:45442 10.134.0.37:443 10.132.0.11:42418 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T10:47:58.143Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 83 0 88 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "9e7e0813-ea5f-4add-aa7e-102507bcd064" "maas.apps.9beba4aa8ba5e1c4c2c4.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.134.0.37:443 10.132.0.11:42424 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T10:47:58.911418Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:47:58.911465Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:47:58.911488Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T10:47:58.911495Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T10:47:58.762Z] "DELETE /v1/api-keys/cc2891c9-92f0-4fc3-beb1-978ca2b79798 HTTP/1.1" 200 - via_upstream - "-" 0 321 149 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "26147d93-fbfa-4fc1-a784-7c71d70133af" "maas.apps.9beba4aa8ba5e1c4c2c4.prod.konflux-ocp-ci.dev" "10.134.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.37:45442 10.134.0.37:443 10.132.0.11:42440 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T10:48:00.754Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 50 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "0bac0fc5-09da-42ca-aa15-805ba5872ffd" "maas.apps.9beba4aa8ba5e1c4c2c4.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.134.0.37:443 10.132.0.11:58582 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T10:48:02.558Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 44 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "db5001cd-d89c-4d23-85f9-34077d008c23" "maas.apps.9beba4aa8ba5e1c4c2c4.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.134.0.37:443 10.132.0.11:58608 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T10:48:02.760323Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:48:02.760378Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:48:02.760402Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T10:48:02.760409Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T10:48:02.657Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 102 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "ebb0a336-ea27-4478-92f3-6b8058e598c9" "maas.apps.9beba4aa8ba5e1c4c2c4.prod.konflux-ocp-ci.dev" "10.134.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.37:45442 10.134.0.37:443 10.132.0.11:58622 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T10:48:03.634156Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:48:03.634195Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:48:03.634210Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:48:03.634215Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T10:48:03.556Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 77 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "400a305d-a9c6-47f2-8e47-24cbb2bf8e96" "maas.apps.9beba4aa8ba5e1c4c2c4.prod.konflux-ocp-ci.dev" "10.134.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.37:45442 10.134.0.37:443 10.132.0.11:58626 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T10:48:12.153601Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-03T10:48:12.229533Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:48:12.229568Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:48:12.229585Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T10:48:12.229589Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T10:48:12.101Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 316 128 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "6f46a213-ace4-4146-89bd-a1871f88fee2" "maas.apps.9beba4aa8ba5e1c4c2c4.prod.konflux-ocp-ci.dev" "10.134.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.37:52282 10.134.0.37:443 10.132.0.11:60632 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T10:48:19.626Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 281 75 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "6cab271b-ad54-488e-ba54-c04bc9724661" "maas.apps.9beba4aa8ba5e1c4c2c4.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.37:45162 10.134.0.37:443 10.132.0.11:34480 - v1-completions-path 2026-11-03T10:48:37.831672Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:48:56.099904Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:50:01.398850Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:50:25.627655Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:51:58.965271Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:55:02.515216Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:55:02.515252Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T10:55:02.515267Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T10:55:02.515272Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T10:55:02.449Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 69 312 65 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "b8123ca7-5bd4-417f-acbc-f1d357d7cade" "maas.apps.9beba4aa8ba5e1c4c2c4.prod.konflux-ocp-ci.dev" "10.134.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.37:42678 10.134.0.37:443 10.132.0.11:34832 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T10:55:02.579Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 110 320 93 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "75bf771d-e36d-452b-8c70-bd9fbcc6bca3" "maas.apps.9beba4aa8ba5e1c4c2c4.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.37:59814 10.134.0.37:443 10.132.0.11:34838 - v1-chat-completions-path 2026-11-03T10:55:57.405100Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:55:57.405135Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:55:57.405150Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:55:57.405155Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T10:55:57.342Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 321 62 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "35d536e6-e8d1-4cc7-baa2-62ea073eb587" "maas.apps.9beba4aa8ba5e1c4c2c4.prod.konflux-ocp-ci.dev" "10.134.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.37:34344 10.134.0.37:443 10.132.0.11:35198 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T11:01:44.563026Z info ads ADS: new connection for node:4 2026-11-03T11:01:44.631399Z info cache read certificate from file resource=file-root:system 2026-11-03T11:07:27.225497Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-03T11:09:18.446302Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T11:09:40.194942Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T11:10:17.724779Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T11:10:56.694694Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T11:11:29.330822Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T11:11:43.013663Z info ads ADS: new connection for node:5 2026-11-03T11:11:43.013868Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T11:11:43.013900Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T11:11:55.001368Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T11:12:26.425423Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T11:15:53.777119Z info ads ADS: "" 5 terminated with stream closed 2026-11-03T11:16:46.813808Z info ads ADS: "" 4 terminated 2026-11-03T11:20:38.134542Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T11:21:53.039429Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T11:24:46.655013Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T11:26:36.660447Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T11:28:24.382567Z info cache event for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt : REMOVE, pushing to proxy 2026-11-03T11:31:38.657038Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=3