2026-10-30T00:50:21.024711Z info FLAG: --concurrency="0" 2026-10-30T00:50:21.024729Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T00:50:21.024733Z info FLAG: --help="false" 2026-10-30T00:50:21.024736Z info FLAG: --log_as_json="false" 2026-10-30T00:50:21.024738Z info FLAG: --log_caller="" 2026-10-30T00:50:21.024740Z info FLAG: --log_output_level="default:info" 2026-10-30T00:50:21.024742Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T00:50:21.024746Z info FLAG: --log_target="[stdout]" 2026-10-30T00:50:21.024748Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T00:50:21.024750Z info FLAG: --outlierLogPath="" 2026-10-30T00:50:21.024752Z info FLAG: --profiling="true" 2026-10-30T00:50:21.024754Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T00:50:21.024756Z info FLAG: --proxyLogLevel="warning" 2026-10-30T00:50:21.024757Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T00:50:21.024759Z info FLAG: --stsPort="0" 2026-10-30T00:50:21.024761Z info FLAG: --templateFile="" 2026-10-30T00:50:21.024763Z info FLAG: --tokenManagerPlugin="" 2026-10-30T00:50:21.024765Z info FLAG: --vklog="0" 2026-10-30T00:50:21.024768Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T00:50:21.024965Z info Proxy role ips=[10.134.0.48] type=router id=maas-default-gateway-openshift-default-6db586dd6c-xw7jh.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T00:50:21.025010Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T00:50:21.026543Z info cpu limit detected as 2, setting concurrency 2026-10-30T00:50:21.026736Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T00:50:21.026748Z info JWT policy is third-party-jwt 2026-10-30T00:50:21.026750Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T00:50:21.028538Z info Starting default Istio SDS Server 2026-10-30T00:50:21.028569Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T00:50:21.028592Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T00:50:21.029161Z info Opening status port 15020 2026-10-30T00:50:21.029688Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T00:50:21.030262Z info sds Starting SDS grpc server 2026-10-30T00:50:21.030280Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T00:50:21.031169Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T00:50:21.031997Z info Starting proxy agent 2026-10-30T00:50:21.032026Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T00:50:21.099848Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T00:50:21.230227Z info ads ADS: new connection for node:1 2026-10-30T00:50:21.230739Z info ads ADS: new connection for node:2 2026-10-30T00:50:21.243774Z info cache generated new workload certificate resourceName=default latency=213.341786ms ttl=23h59m59.756228765s 2026-10-30T00:50:21.243804Z info cache Root cert has changed, start rotating root cert 2026-10-30T00:50:21.243841Z info cache returned workload trust anchor from cache ttl=23h59m59.75615918s 2026-10-30T00:50:21.243854Z info cache returned workload certificate from cache ttl=23h59m59.756146226s 2026-10-30T00:50:21.243977Z info cache returned workload trust anchor from cache ttl=23h59m59.756024752s 2026-10-30T00:50:21.244297Z info cache returned workload trust anchor from cache ttl=23h59m59.755705006s 2026-10-30T00:50:21.325736Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T00:50:29.224519Z info Readiness succeeded in 8.202022605s 2026-10-30T00:50:29.225450Z info Envoy proxy is ready 2026-10-30T00:50:51.476530Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T00:50:51.476578Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T00:50:51.476601Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T00:50:51.476608Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T00:50:51.384Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 568 91 - "34.228.250.194,10.132.0.21" "curl/7.76.1" "07c573ac-873d-4379-a8b0-c3428ad2bcf1" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "10.134.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.48:60258 10.134.0.48:443 10.132.0.21:36568 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T00:53:46.145089Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T00:53:46.145140Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T00:53:46.145162Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T00:53:46.145168Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T00:53:46.033Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 111 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "9affbea0-0c32-499a-9558-9d4d164047ae" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "10.134.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.48:57324 10.134.0.48:443 10.132.0.21:49508 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T00:53:46.437594Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T00:53:46.437630Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T00:53:46.437643Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T00:53:46.437648Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T00:53:46.835588Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T00:53:46.835621Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T00:53:46.835635Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T00:53:46.835639Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T00:53:47.129828Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T00:53:47.129872Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T00:53:47.129892Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T00:53:47.129898Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T00:53:46.282Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 48 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "77fd4ab6-43e4-4fc2-b7e9-cf5cc73a8b6f" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.134.0.48:443 10.132.0.21:49516 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T00:53:46.351Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 85 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "a437c6f2-8ea6-46bb-a5da-7d07051093a9" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "10.134.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.48:57334 10.134.0.48:443 10.132.0.21:49528 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T00:53:46.720Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 114 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "d288e16a-1760-41fd-8bf4-6c0e671818ef" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "10.134.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.48:57324 10.134.0.48:443 10.132.0.21:49534 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T00:53:46.921Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 65 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "760cc641-427f-4490-9af8-8b39a9d8e141" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.134.0.48:443 10.132.0.21:49536 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T00:53:47.034Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 95 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "f96ecef3-4236-468f-82ab-77388bb3e218" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "10.134.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.48:57324 10.134.0.48:443 10.132.0.21:49548 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T00:53:47.505605Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T00:53:47.505642Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T00:53:47.505656Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T00:53:47.505660Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T00:53:47.631467Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T00:53:47.631506Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T00:53:47.631525Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T00:53:47.631532Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T00:53:47.807358Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T00:53:47.807462Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T00:53:47.807516Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T00:53:47.807547Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T00:53:47.421Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 84 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "fc71ebc0-fc0b-42e2-abb5-d194dd4eb0d4" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "10.134.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.48:57334 10.134.0.48:443 10.132.0.21:49562 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T00:53:47.524Z] "DELETE /v1/api-keys/707e21e6-7536-4240-9088-3a3c2abbfa55 HTTP/1.1" 200 - via_upstream - "-" 0 321 107 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "151e5bb0-94ae-40a6-93e2-fd52138392d0" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "10.134.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.48:57334 10.134.0.48:443 10.132.0.21:49568 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T00:53:47.706Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 317 100 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "6273cbab-bb1d-42d1-8ef3-23169689b324" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "10.134.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.48:57334 10.134.0.48:443 10.132.0.21:49578 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T00:53:48.296123Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-10-30T00:53:48.301897Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T00:53:48.301983Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T00:53:48.302023Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T00:53:48.302052Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T00:53:48.182Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 119 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "4b44d9c9-4213-4859-9bbd-80ef0822d5a1" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "10.134.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.48:57334 10.134.0.48:443 10.132.0.21:49592 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T00:53:52.833Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 41 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "049242db-64a5-4f4c-b0e7-fc0b680d784d" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.48:443 10.132.0.21:49298 - v1-completions-path 2026-10-30T00:53:53.447835Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T00:53:53.447872Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T00:53:53.447876Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T00:53:53.447881Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-10-30T00:53:53.298Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 76 77 149 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "04501c45-66b0-4967-9ea6-18bea653fc73" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.48:443 10.132.0.21:49304 - v1-completions-path [2026-10-30T00:53:53.931Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 281 85 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "0b23d2dc-7e84-468a-b70f-30e7f76fa625" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "10.132.0.48:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.48:36358 10.134.0.48:443 10.132.0.21:49314 - v1-completions-path 2026-10-30T00:53:59.908718Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T00:53:59.908754Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T00:53:59.908767Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T00:53:59.908772Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T00:53:59.835Z] "DELETE /v1/api-keys/9a8abe02-5a0f-4762-9258-acaee988f425 HTTP/1.1" 200 - via_upstream - "-" 0 321 73 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "1e2d99ba-a86e-4d3a-adc4-84be561fa3a4" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "10.134.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.48:57334 10.134.0.48:443 10.132.0.21:49334 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T00:54:08.528779Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T00:54:24.240189Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T00:57:27.489956Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T00:57:27.490157Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T00:57:27.423Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 135 3095 66 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "a84e12ee-23b1-42ae-b031-58b1a4651993" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "10.132.0.48:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.48:46554 10.134.0.48:443 10.132.0.21:41260 - v1-chat-completions-path 2026-10-30T00:59:55.040900Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:00:20.186441Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:00:37.481049Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:01:01.896653Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:02:02.257342Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:02:41.612749Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:19:45.508444Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-10-30T01:28:53.072450Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T01:28:53.072490Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T01:28:53.072506Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T01:28:53.072511Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T01:28:52.993Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 313 78 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "2f1334a7-965a-4ced-8e17-420b93232519" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "10.134.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.48:52942 10.134.0.48:443 10.132.0.21:34784 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T01:28:54.078507Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:29:17.218072Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:29:28.821382Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T01:29:28.821416Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T01:29:28.821427Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T01:29:28.821432Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T01:29:28.740Z] "POST /v1/api-keys HTTP/1.1" 400 - via_upstream - "-" 80 94 81 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "7ca12df2-9416-46d6-934a-c5766ee137d8" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "10.134.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.48:51674 10.134.0.48:443 10.132.0.21:37030 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T01:29:58.825988Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:30:16.014053Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:30:20.934377Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T01:30:26.888Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 97 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "6e966ce3-6440-4416-8515-cc5151027470" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "10.132.0.47:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.48:43054 10.134.0.48:443 10.132.0.21:59322 - v1-completions-path 2026-10-30T01:30:39.261829Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:30:41.639041Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:30:44.801840Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:30:54.574826Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:31:18.060703Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:31:28.288842Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:31:32.425743Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:31:36.483876Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T01:31:36.483911Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T01:31:36.483925Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T01:31:36.483930Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T01:31:36.399Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 317 84 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "06e3d13a-e664-48ca-b245-920bdc87dc77" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "10.134.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.48:44874 10.134.0.48:443 10.132.0.21:36216 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T01:31:36.864Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 67 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "450c123d-cc30-4b12-9004-cb21f4893c4f" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "10.132.0.47:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.48:38138 10.134.0.48:443 10.132.0.21:36224 - v1-completions-path 2026-10-30T01:31:49.757825Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:31:51.918538Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:31:59.283462Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-10-30T01:31:59.205Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 74 293 79 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "d6e7c7d3-9ba3-4ffc-8687-80699f79b47d" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "10.134.0.44:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.134.0.48:36728 10.134.0.48:443 10.132.0.21:32898 - v1-completions-path 2026-10-30T01:32:11.205072Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:32:15.480917Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:32:31.956503Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:32:48.902524Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:32:59.585910Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:33:02.662372Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:33:21.532466Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:33:24.151941Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:33:45.203409Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:33:47.850559Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:35:22.659029Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:35:26.045490Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:36:55.634788Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:37:10.638405Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:37:12.770493Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T01:37:13.374Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 281 82 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "50df05c3-18a0-4b97-859c-6ccd88425627" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "10.132.0.48:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.48:46554 10.134.0.48:443 10.132.0.21:55716 - v1-completions-path 2026-10-30T01:37:27.282785Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:38:40.343876Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-10-30T01:38:40.343979Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T01:38:40.344000Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T01:38:40.344013Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T01:38:40.344021Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T01:38:40.289Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 54 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "95f2833d-3947-450f-a631-51b35f6b75a4" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "10.134.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.48:50310 10.134.0.48:443 10.132.0.21:54938 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T01:38:44.877356Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:39:07.140708Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:39:20.326117Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T01:39:20.326166Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T01:39:20.326225Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T01:39:20.326234Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T01:39:20.246Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 80 322 79 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "c0f58809-c775-4865-b7e1-e690adb85026" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "10.134.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.48:39610 10.134.0.48:443 10.132.0.21:35840 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T01:39:22.355642Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:39:34.230916Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:39:38.445159Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:39:49.839544Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:39:53.430809Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:40:13.336397Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:40:22.726353Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:40:30.957634Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T01:40:37.047Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 429 - - "-" 61 18 76 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "1aa92d60-34b5-4deb-b366-198ad96ee71b" "maas.apps.b9bf3e24ff1e8e8fb6fd.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.48:443 10.132.0.21:55124 - v1-catch-all-path 2026-10-30T01:40:44.425095Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:40:46.583975Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:41:01.391487Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T01:41:12.278881Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8