2026-11-02T21:45:05.164820Z info FLAG: --concurrency="0" 2026-11-02T21:45:05.164846Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-02T21:45:05.164850Z info FLAG: --help="false" 2026-11-02T21:45:05.164852Z info FLAG: --log_as_json="false" 2026-11-02T21:45:05.164855Z info FLAG: --log_caller="" 2026-11-02T21:45:05.164857Z info FLAG: --log_output_level="default:info" 2026-11-02T21:45:05.164859Z info FLAG: --log_stacktrace_level="default:none" 2026-11-02T21:45:05.164864Z info FLAG: --log_target="[stdout]" 2026-11-02T21:45:05.164866Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-02T21:45:05.164868Z info FLAG: --outlierLogPath="" 2026-11-02T21:45:05.164870Z info FLAG: --profiling="true" 2026-11-02T21:45:05.164873Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-02T21:45:05.164874Z info FLAG: --proxyLogLevel="warning" 2026-11-02T21:45:05.164877Z info FLAG: --serviceCluster="istio-proxy" 2026-11-02T21:45:05.164879Z info FLAG: --stsPort="0" 2026-11-02T21:45:05.164881Z info FLAG: --templateFile="" 2026-11-02T21:45:05.164883Z info FLAG: --tokenManagerPlugin="" 2026-11-02T21:45:05.164885Z info FLAG: --vklog="0" 2026-11-02T21:45:05.164888Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-02T21:45:05.165061Z info Proxy role ips=[10.133.0.41] type=router id=maas-default-gateway-openshift-default-65c448b547-mcx5l.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-02T21:45:05.165115Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-02T21:45:05.166280Z info cpu limit detected as 2, setting concurrency 2026-11-02T21:45:05.166481Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-02T21:45:05.166492Z info JWT policy is third-party-jwt 2026-11-02T21:45:05.166495Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-02T21:45:05.168246Z info Starting default Istio SDS Server 2026-11-02T21:45:05.168279Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-02T21:45:05.168297Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-02T21:45:05.168510Z info Opening status port 15020 2026-11-02T21:45:05.169067Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-02T21:45:05.171658Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-02T21:45:05.174010Z info Starting proxy agent 2026-11-02T21:45:05.174772Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-02T21:45:05.174041Z info sds Starting SDS grpc server 2026-11-02T21:45:05.191309Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-02T21:45:05.243180Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-02T21:45:05.336485Z info cache generated new workload certificate resourceName=default latency=167.006163ms ttl=23h59m59.663518679s 2026-11-02T21:45:05.336528Z info cache Root cert has changed, start rotating root cert 2026-11-02T21:45:05.336566Z info cache returned workload trust anchor from cache ttl=23h59m59.663435088s 2026-11-02T21:45:05.407427Z info ads ADS: new connection for node:1 2026-11-02T21:45:05.407548Z info cache returned workload certificate from cache ttl=23h59m59.592454453s 2026-11-02T21:45:05.408155Z info ads ADS: new connection for node:2 2026-11-02T21:45:05.408237Z info cache returned workload trust anchor from cache ttl=23h59m59.591764357s 2026-11-02T21:45:05.506976Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:45:13.413011Z info Readiness succeeded in 8.250783808s 2026-11-02T21:45:13.413479Z info Envoy proxy is ready 2026-11-02T21:50:24.002014Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:50:24.002052Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:50:24.002066Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T21:50:24.002071Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-02T21:50:23.880Z] "DELETE /v1/api-keys/040bbcb5-f4b2-47aa-b4b7-7982b99094c3 HTTP/1.1" 200 - via_upstream - "-" 0 321 121 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "9d583d30-f94c-4f90-8b27-30d33b218d2c" "maas.apps.0f8d2781715f7880749e.prod.konflux-ocp-ci.dev" "10.134.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.41:40594 10.133.0.41:443 10.132.0.14:48806 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T21:50:25.401176Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:50:25.401208Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:50:25.401221Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T21:50:25.401226Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-02T21:50:25.261Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 139 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "1f7ec0c5-7c83-436b-8687-21551268025d" "maas.apps.0f8d2781715f7880749e.prod.konflux-ocp-ci.dev" "10.134.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.41:40594 10.133.0.41:443 10.132.0.14:48820 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T21:50:25.717860Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T21:50:25.717896Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T21:50:25.717911Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T21:50:25.717916Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-02T21:50:25.592Z] "DELETE /v1/api-keys/25b50a25-c955-4e02-8189-8ff6a287c9f4 HTTP/1.1" 200 - via_upstream - "-" 0 321 125 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "87e46165-5ff3-4126-a9b5-715d926a57d2" "maas.apps.0f8d2781715f7880749e.prod.konflux-ocp-ci.dev" "10.134.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.41:40596 10.133.0.41:443 10.132.0.14:48836 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T21:50:28.301Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 72 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "e07b4f81-1f32-4370-bd64-1e7dc9d56df9" "maas.apps.0f8d2781715f7880749e.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.41:42070 10.133.0.41:443 10.132.0.14:48854 - v1-completions-path [2026-11-02T21:50:28.567Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 65 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "63615a62-aa6a-4931-a499-b7c11a5d6b10" "maas.apps.0f8d2781715f7880749e.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.41:42082 10.133.0.41:443 10.132.0.14:48864 - v1-completions-path [2026-11-02T21:50:33.983Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 284 125 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "395b7289-b417-474a-b4f3-a17f0d24c6da" "maas.apps.0f8d2781715f7880749e.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.41:42082 10.133.0.41:443 10.132.0.14:60084 - v1-completions-path 2026-11-02T21:50:34.786469Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:50:43.579200Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:50:43.579235Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:50:43.579248Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T21:50:43.579253Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-02T21:50:43.466Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 317 112 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "77698fc3-ea86-42fe-b244-09e01daf503f" "maas.apps.0f8d2781715f7880749e.prod.konflux-ocp-ci.dev" "10.134.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.41:40594 10.133.0.41:443 10.132.0.14:39904 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T21:50:45.198Z] "POST /maas-api/internal/v1/api-keys/cleanup HTTP/1.1" 404 NR route_not_found - "-" 2 0 7 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "36d3e924-3f02-4877-9fb1-9299c73ccc4d" "maas.apps.0f8d2781715f7880749e.prod.konflux-ocp-ci.dev" "-" - - 10.133.0.41:443 10.132.0.14:39914 - - 2026-11-02T21:50:52.610854Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:56:17.626042Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T21:56:17.626197Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T21:56:17.626259Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T21:56:17.626303Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T21:56:17.156Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 321 469 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "e3e66cf2-1ad8-44e9-b626-d101ac204560" "maas.apps.0f8d2781715f7880749e.prod.konflux-ocp-ci.dev" "10.134.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.41:57360 10.133.0.41:443 10.132.0.14:54156 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T21:56:23.008681Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T21:56:23.008737Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T21:56:23.008759Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T21:56:23.008767Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T21:56:22.743Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 322 265 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "cb51009d-af2c-41df-88ad-97aba5caf298" "maas.apps.0f8d2781715f7880749e.prod.konflux-ocp-ci.dev" "10.134.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.41:57360 10.133.0.41:443 10.132.0.14:38868 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T21:56:23.490779Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:56:23.490824Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:56:23.490844Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T21:56:23.490851Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:56:23.491164Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 2026-11-02T21:56:23.491239Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-02T21:56:23.491247Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-02T21:56:23.491251Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-02T21:56:23.491256Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-11-02T21:56:23.206Z] "POST /v1/completions HTTP/1.1" 404 - via_upstream - "-" 94 126 284 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "55fb566d-573a-4aba-9504-f1d090a6abe9" "maas.apps.0f8d2781715f7880749e.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.41:42070 10.133.0.41:443 10.132.0.14:38882 - v1-completions-path 2026-11-02T21:59:36.964947Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:00:48.698371Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:01:38.155666Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:02:17.281659Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:03:36.857069Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:05:10.005642Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:05:52.968760Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:07:33.728314Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:13:05.441958Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:14:36.088671Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:15:33.265510Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-02T22:19:56.876469Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=3 2026-11-02T22:22:58.010361Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:23:24.154384Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:24:19.768753Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T22:24:25.568Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 75 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "a8f616be-cd58-4eb2-9e7c-3a1828f8062c" "maas.apps.0f8d2781715f7880749e.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.41:56688 10.133.0.41:443 10.132.0.14:60928 - v1-completions-path 2026-11-02T22:24:41.018937Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:24:45.437080Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:25:12.281158Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:25:30.834584Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T22:25:41.004Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 298 62 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "911cc1d8-b00b-4197-88d2-911265054f97" "maas.apps.0f8d2781715f7880749e.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.41:57758 10.133.0.41:443 10.132.0.14:36874 - v1-completions-path 2026-11-02T22:25:44.148374Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-11-02T22:25:44.096Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 54 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "c810f0cc-5c08-427b-87c5-bccf4c2e7c14" "maas.apps.0f8d2781715f7880749e.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.41:57758 10.133.0.41:443 10.132.0.14:36890 - v1-completions-path 2026-11-02T22:25:44.409217Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-02T22:25:44.348Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 61 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "55715f39-b343-479e-9367-0aa3300c9a15" "maas.apps.0f8d2781715f7880749e.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.41:56688 10.133.0.41:443 10.132.0.14:36894 - v1-completions-path 2026-11-02T22:26:02.560912Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:26:13.149359Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:26:19.368849Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:26:30.108526Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:26:30.108561Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:26:30.108575Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:26:30.108580Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T22:26:29.998Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 317 110 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "c7a517f0-7023-4567-9fc2-785194da54bd" "maas.apps.0f8d2781715f7880749e.prod.konflux-ocp-ci.dev" "10.134.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.41:45452 10.133.0.41:443 10.132.0.14:47522 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T22:26:47.842231Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:26:52.178204Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:27:15.723150Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:27:21.957801Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:27:48.006938Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:28:09.974008Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:28:23.243415Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:28:55.544156Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:29:16.952045Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:29:27.658953Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:31:43.292897Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:31:43.292931Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:31:43.292947Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T22:31:43.292952Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-02T22:31:43.178Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 80 322 114 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "e5f68e67-dadb-4899-ad00-ad0ec91299b6" "maas.apps.0f8d2781715f7880749e.prod.konflux-ocp-ci.dev" "10.134.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.41:47950 10.133.0.41:443 10.132.0.14:47878 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T22:33:27.296854Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:33:44.753814Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:33:49.181572Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:34:13.842056Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:36:19.849271Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:37:02.561941Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:37:33.296619Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:37:48.759532Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:37:58.431077Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:37:58.431110Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:37:58.431143Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:37:58.431148Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T22:37:58.320Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 84 327 110 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "7a611c67-f32e-4f4e-a616-68052975682a" "maas.apps.0f8d2781715f7880749e.prod.konflux-ocp-ci.dev" "10.134.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.41:39280 10.133.0.41:443 10.132.0.14:37294 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T22:37:59.864781Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:38:12.196180Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:38:16.992249Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T22:38:28.547Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 61 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "4715e03a-46c0-4e72-b476-0385a351a0c6" "maas.apps.0f8d2781715f7880749e.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.41:443 10.132.0.14:38534 - v1-completions-path 2026-11-02T22:38:42.586354Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:42:16.369341Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:42:29.395051Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:42:29.395084Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:42:29.395098Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:42:29.395103Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T22:42:29.279Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 323 115 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "87817a6d-7622-4272-8339-6f6f803e0f6b" "maas.apps.0f8d2781715f7880749e.prod.konflux-ocp-ci.dev" "10.134.0.156:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.41:60664 10.133.0.41:443 10.132.0.14:53234 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T22:42:30.367001Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-11-02T22:42:30.296Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 91 3616 70 - "44.212.242.249,10.132.0.14" "python-requests/2.34.2" "d7c9458b-80c6-4339-81e1-02b228f0317c" "maas.apps.0f8d2781715f7880749e.prod.konflux-ocp-ci.dev" "10.133.0.33:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.41:56894 10.133.0.41:443 10.132.0.14:53246 - v1-catch-all-model-routing 2026-11-02T22:42:34.509157Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:42:55.356392Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:42:59.851794Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:43:15.738395Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:43:27.399553Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8