2026-11-02T08:13:30.594873Z info FLAG: --concurrency="0" 2026-11-02T08:13:30.594910Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-02T08:13:30.594914Z info FLAG: --help="false" 2026-11-02T08:13:30.594917Z info FLAG: --log_as_json="false" 2026-11-02T08:13:30.594919Z info FLAG: --log_caller="" 2026-11-02T08:13:30.594921Z info FLAG: --log_output_level="default:info" 2026-11-02T08:13:30.594923Z info FLAG: --log_stacktrace_level="default:none" 2026-11-02T08:13:30.594927Z info FLAG: --log_target="[stdout]" 2026-11-02T08:13:30.594930Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-02T08:13:30.594932Z info FLAG: --outlierLogPath="" 2026-11-02T08:13:30.594934Z info FLAG: --profiling="true" 2026-11-02T08:13:30.594936Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-02T08:13:30.594938Z info FLAG: --proxyLogLevel="warning" 2026-11-02T08:13:30.594940Z info FLAG: --serviceCluster="istio-proxy" 2026-11-02T08:13:30.594943Z info FLAG: --stsPort="0" 2026-11-02T08:13:30.594944Z info FLAG: --templateFile="" 2026-11-02T08:13:30.594946Z info FLAG: --tokenManagerPlugin="" 2026-11-02T08:13:30.594949Z info FLAG: --vklog="0" 2026-11-02T08:13:30.594952Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-02T08:13:30.595156Z info Proxy role ips=[10.132.0.33] type=router id=maas-default-gateway-openshift-default-769df7846c-tskjl.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-02T08:13:30.595293Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-02T08:13:30.596786Z info cpu limit detected as 2, setting concurrency 2026-11-02T08:13:30.597049Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-02T08:13:30.597064Z info JWT policy is third-party-jwt 2026-11-02T08:13:30.597068Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-02T08:13:30.599503Z info Starting default Istio SDS Server 2026-11-02T08:13:30.599543Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-02T08:13:30.599607Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-02T08:13:30.599958Z info Opening status port 15020 2026-11-02T08:13:30.600780Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-02T08:13:30.601991Z info sds Starting SDS grpc server 2026-11-02T08:13:30.602007Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-02T08:13:30.602791Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-02T08:13:30.604040Z info Starting proxy agent 2026-11-02T08:13:30.604086Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-02T08:13:30.671574Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-02T08:13:30.755128Z info cache generated new workload certificate resourceName=default latency=152.89899ms ttl=23h59m59.244877598s 2026-11-02T08:13:30.755267Z info cache Root cert has changed, start rotating root cert 2026-11-02T08:13:30.755343Z info cache returned workload trust anchor from cache ttl=23h59m59.244659653s 2026-11-02T08:13:30.819219Z info ads ADS: new connection for node:1 2026-11-02T08:13:30.819304Z info cache returned workload certificate from cache ttl=23h59m59.18069789s 2026-11-02T08:13:30.819817Z info ads ADS: new connection for node:2 2026-11-02T08:13:30.819902Z info cache returned workload trust anchor from cache ttl=23h59m59.180098892s 2026-11-02T08:13:30.865114Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:13:36.984473Z info Readiness succeeded in 6.392600239s 2026-11-02T08:13:36.984786Z info Envoy proxy is ready 2026-11-02T08:14:35.451759Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:14:38.840837Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:15:38.592485Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:15:43.783474Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:15:49.258770Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:15:53.600757Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:15:59.804850Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:16:07.834645Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:16:41.988404Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T08:16:41.988446Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T08:16:41.988463Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T08:16:41.988468Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-02T08:16:41.889Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 99 - "44.212.242.249,10.132.0.16" "curl/7.76.1" "1aded23f-a15d-4cb0-baaa-2080a1d6c809" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.33:46730 10.132.0.33:443 10.132.0.16:35342 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T08:16:42.873Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 507 77 - "44.212.242.249,10.132.0.16" "curl/7.76.1" "0e18915b-4271-4757-81a1-f6f3891fcba4" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "10.133.0.46:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.33:45200 10.132.0.33:443 10.132.0.16:35352 - v1-model-routing [2026-11-02T08:16:43.417Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 61 - "44.212.242.249,10.132.0.16" "curl/7.76.1" "0a52b5fd-042d-4395-b3ad-2c9c84290a2e" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.33:443 10.132.0.16:35354 - v1-model-routing 2026-11-02T08:16:45.458469Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-02T08:16:45.458498Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-02T08:16:45.458501Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-02T08:16:45.458505Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 2026-11-02T08:16:45.779293Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T08:16:45.779330Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T08:16:45.779346Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T08:16:45.779351Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-02T08:16:45.439Z] "POST /v1/chat/completions HTTP/1.1" 401 - - "-" 122 88 18 - "44.212.242.249,10.132.0.16" "curl/7.76.1" "c1d7811a-c4b4-41fc-844c-a87aec07b6f1" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.33:443 10.132.0.16:35362 - v1-model-routing [2026-11-02T08:16:45.670Z] "DELETE /v1/api-keys/9e94961a-aea9-4e63-aaef-56abd8b408d8 HTTP/1.1" 200 - via_upstream - "-" 0 365 108 - "44.212.242.249,10.132.0.16" "curl/7.76.1" "203773aa-4566-41c2-aa1f-e8e2a3de285f" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.33:46730 10.132.0.33:443 10.132.0.16:35372 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T08:28:10.656170Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:28:34.724786Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T08:29:47.944Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 83 0 52 - "44.212.242.249,10.132.0.16" "python-requests/2.34.2" "05112314-35af-43b7-9837-efaea2049e98" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.132.0.33:443 10.132.0.16:41116 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T08:29:49.710807Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:29:49.710856Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:29:49.710876Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T08:29:49.710883Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-02T08:29:49.315Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 92 - "44.212.242.249,10.132.0.16" "python-requests/2.34.2" "a490f8f0-7715-44a6-85de-018805f0f050" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.132.0.33:443 10.132.0.16:41118 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T08:29:49.593Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 117 - "44.212.242.249,10.132.0.16" "python-requests/2.34.2" "72d0aa89-e8b7-4523-bb7b-3f8cb02d0f95" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.33:47654 10.132.0.33:443 10.132.0.16:41126 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T08:29:50.544097Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:29:50.544131Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:29:50.544176Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:29:50.544181Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T08:29:50.433Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 110 - "44.212.242.249,10.132.0.16" "python-requests/2.34.2" "93aec992-18c6-45c4-ac2a-e324051fe4ae" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.33:47654 10.132.0.33:443 10.132.0.16:34248 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T08:29:51.422796Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:29:51.422835Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:29:51.422852Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:29:51.422857Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T08:29:51.307Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 115 - "44.212.242.249,10.132.0.16" "python-requests/2.34.2" "cdfd9901-3f00-42bc-a017-16c803245a38" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.33:47654 10.132.0.33:443 10.132.0.16:34260 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T08:29:57.400125Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T08:29:57.400274Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T08:29:57.400324Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T08:29:57.400365Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-02T08:29:57.190Z] "DELETE /v1/api-keys/6f019c96-33a8-4fe6-9502-17cbf5c6b24f HTTP/1.1" 200 - via_upstream - "-" 0 321 209 - "44.212.242.249,10.132.0.16" "python-requests/2.34.2" "ac89d28a-1da1-48b1-9ad2-04c31056bb0a" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.33:53540 10.132.0.33:443 10.132.0.16:34276 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T08:29:58.032529Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T08:29:58.032576Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T08:29:58.032598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T08:29:58.032606Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-02T08:29:57.769Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 263 - "44.212.242.249,10.132.0.16" "python-requests/2.34.2" "c7534ad2-3f32-4e6c-9aad-2b00afb05cce" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.33:53540 10.132.0.33:443 10.132.0.16:34280 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T08:29:58.953706Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-02T08:29:58.953802Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:29:58.953823Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:29:58.953840Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T08:29:58.953846Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:29:59.185919Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-02T08:29:59.185948Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-02T08:29:59.185952Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-02T08:29:59.185956Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-11-02T08:29:58.717Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 236 - "44.212.242.249,10.132.0.16" "python-requests/2.34.2" "c4bb97a5-a591-4eb8-b6f2-d3bfc8365a01" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.33:47654 10.132.0.33:443 10.132.0.16:34290 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T08:29:59.142Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 76 77 43 - "44.212.242.249,10.132.0.16" "python-requests/2.34.2" "d25c3ff4-0382-4312-8c3b-1b5796dc137f" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.33:443 10.132.0.16:34300 - v1-completions-path 2026-11-02T08:30:03.240100Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:30:03.240174Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:30:03.240202Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:30:03.240210Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T08:30:03.068Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 171 - "44.212.242.249,10.132.0.16" "python-requests/2.34.2" "1e621839-b051-4269-a177-82b3719cb3d1" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.33:47654 10.132.0.33:443 10.132.0.16:53220 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T08:30:03.567106Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:30:03.567173Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:30:03.567190Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:30:03.567195Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T08:30:04.069639Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:30:04.069688Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:30:04.069714Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:30:04.069722Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T08:30:03.420Z] "DELETE /v1/api-keys/b932b994-7062-4b7a-9e66-b4a4055caa2f HTTP/1.1" 200 - via_upstream - "-" 0 321 147 - "44.212.242.249,10.132.0.16" "python-requests/2.34.2" "329c3dd7-412d-4fde-b9f4-24ff1b069dbc" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.33:47654 10.132.0.33:443 10.132.0.16:53222 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T08:30:03.934Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 135 - "44.212.242.249,10.132.0.16" "python-requests/2.34.2" "9339c0d5-ee28-40f0-b34d-3bb92f081ac3" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.33:47654 10.132.0.33:443 10.132.0.16:53238 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T08:30:05.341653Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-11-02T08:30:05.347004Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T08:30:05.347109Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T08:30:05.347189Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T08:30:05.347230Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-02T08:30:04.579Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 103 - "44.212.242.249,10.132.0.16" "python-requests/2.34.2" "65c85a9f-232d-41a9-84ca-9095f821382b" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.132.0.33:443 10.132.0.16:53258 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T08:30:04.981Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 289 92 - "44.212.242.249,10.132.0.16" "python-requests/2.34.2" "12151e61-b22b-46c4-9346-8adfb36b2155" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "10.133.0.46:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.33:45200 10.132.0.33:443 10.132.0.16:53262 - v1-completions-path [2026-11-02T08:30:05.184Z] "DELETE /v1/api-keys/e6258a07-7ffa-42f7-8759-1d4127bc736b HTTP/1.1" 200 - via_upstream - "-" 0 321 162 - "44.212.242.249,10.132.0.16" "python-requests/2.34.2" "aec6fcd3-5572-45be-818b-23dace285879" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.33:53540 10.132.0.33:443 10.132.0.16:53276 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T08:30:07.770Z] "POST /maas-api/internal/v1/api-keys/validate HTTP/1.1" 404 NR route_not_found - "-" 31 0 8 - "44.212.242.249,10.132.0.16" "python-requests/2.34.2" "106fd238-e5a9-4e49-a94a-89a41388ff0b" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "-" - - 10.132.0.33:443 10.132.0.16:53304 - - 2026-11-02T08:30:12.277834Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: gRPC status code is not OK thread=24 [2026-11-02T08:30:10.275Z] "GET /maas-api/v1/models HTTP/1.1" 500 - - "-" 0 23 2003 - "44.212.242.249,10.132.0.16" "python-requests/2.34.2" "8546d48f-e564-4f7a-86a5-19ae422cb1fb" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.132.0.33:443 10.132.0.16:49030 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T08:39:50.240296Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T08:39:50.240330Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T08:39:50.240344Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T08:39:50.240348Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-02T08:39:50.225Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 20 15 - "44.212.242.249,10.132.0.16" "python-requests/2.34.2" "c727f3cd-76b7-497b-a05b-4de790100c47" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.33:42330 10.132.0.33:443 10.132.0.16:49276 - odh-ai-gateway-infra.maas-api-route.4 2026-11-02T08:40:32.126403Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:41:25.990826Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:42:44.761509Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:45:43.988659Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-02T08:46:02.106656Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:46:02.106694Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:46:02.106710Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:46:02.106715Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T08:46:01.937Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 69 311 169 - "44.212.242.249,10.132.0.16" "python-requests/2.34.2" "6a9f108b-e09c-40e7-a224-328437226881" "maas.apps.65cabb995543feab5d8c.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.33:54328 10.132.0.33:443 10.132.0.16:49650 - odh-ai-gateway-infra.maas-api-route.3