2026-11-04T01:11:19.900029Z info FLAG: --concurrency="0" 2026-11-04T01:11:19.900048Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-04T01:11:19.900052Z info FLAG: --help="false" 2026-11-04T01:11:19.900055Z info FLAG: --log_as_json="false" 2026-11-04T01:11:19.900057Z info FLAG: --log_caller="" 2026-11-04T01:11:19.900059Z info FLAG: --log_output_level="default:info" 2026-11-04T01:11:19.900061Z info FLAG: --log_stacktrace_level="default:none" 2026-11-04T01:11:19.900066Z info FLAG: --log_target="[stdout]" 2026-11-04T01:11:19.900068Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-04T01:11:19.900070Z info FLAG: --outlierLogPath="" 2026-11-04T01:11:19.900072Z info FLAG: --profiling="true" 2026-11-04T01:11:19.900074Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-04T01:11:19.900076Z info FLAG: --proxyLogLevel="warning" 2026-11-04T01:11:19.900078Z info FLAG: --serviceCluster="istio-proxy" 2026-11-04T01:11:19.900080Z info FLAG: --stsPort="0" 2026-11-04T01:11:19.900082Z info FLAG: --templateFile="" 2026-11-04T01:11:19.900084Z info FLAG: --tokenManagerPlugin="" 2026-11-04T01:11:19.900086Z info FLAG: --vklog="0" 2026-11-04T01:11:19.900089Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-04T01:11:19.900228Z info Proxy role ips=[10.133.0.35] type=router id=maas-default-gateway-openshift-default-848f6bf8bc-kvnxf.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-04T01:11:19.900275Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-04T01:11:19.901645Z info cpu limit detected as 2, setting concurrency 2026-11-04T01:11:19.901839Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-04T01:11:19.901849Z info JWT policy is third-party-jwt 2026-11-04T01:11:19.901853Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-04T01:11:19.903285Z info Starting default Istio SDS Server 2026-11-04T01:11:19.903303Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-04T01:11:19.903320Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-04T01:11:19.903744Z info Opening status port 15020 2026-11-04T01:11:19.904488Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-04T01:11:19.905037Z info sds Starting SDS grpc server 2026-11-04T01:11:19.905052Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-04T01:11:19.905717Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-04T01:11:19.906488Z info Starting proxy agent 2026-11-04T01:11:19.906517Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-04T01:11:19.969661Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-04T01:11:19.994386Z info cache generated new workload certificate resourceName=default latency=89.255769ms ttl=23h59m59.005617608s 2026-11-04T01:11:19.994418Z info cache Root cert has changed, start rotating root cert 2026-11-04T01:11:19.994447Z info cache returned workload trust anchor from cache ttl=23h59m59.005553082s 2026-11-04T01:11:20.065122Z info ads ADS: new connection for node:1 2026-11-04T01:11:20.065183Z info cache returned workload certificate from cache ttl=23h59m58.934818763s 2026-11-04T01:11:20.065855Z info ads ADS: new connection for node:2 2026-11-04T01:11:20.065929Z info cache returned workload trust anchor from cache ttl=23h59m58.934072123s 2026-11-04T01:11:20.152978Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:11:28.605308Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:11:29.538942Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-04T01:11:30.000478Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-04T01:11:31.433548Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:11:31.467537Z info Readiness succeeded in 11.569771558s 2026-11-04T01:11:31.468070Z info Envoy proxy is ready 2026-11-04T01:11:31.556582Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:12:07.272957Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:12:07.272992Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:12:07.273005Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:12:07.273011Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-04T01:12:07.253Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 91 19 - "52.71.36.72,10.132.0.10" "curl/7.76.1" "1b961996-2cab-4066-bfd8-784419083bd0" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.35:53046 10.133.0.35:443 10.132.0.10:56500 - odh-ai-gateway-infra.maas-api-route.4 2026-11-04T01:14:31.463056Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:14:31.463095Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:14:31.463109Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T01:14:31.463114Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:14:31.759310Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:14:31.759375Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:14:31.759393Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:14:31.759398Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T01:14:31.372Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 84 326 90 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "deac2649-30f8-43da-b135-2dd0040fe46f" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.35:32984 10.133.0.35:443 10.132.0.10:54800 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T01:14:31.695Z] "DELETE /v1/api-keys/2ea593cf-df4b-47a7-a18a-6ee78116eb7a HTTP/1.1" 200 - via_upstream - "-" 0 370 64 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "8a3121dc-18f2-4db2-8e39-c0a0a7b7ded2" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.35:32984 10.133.0.35:443 10.132.0.10:54806 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T01:14:32.883833Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:14:32.883868Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:14:32.883886Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T01:14:32.883892Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:14:33.406390Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:14:33.406423Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:14:33.406435Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T01:14:33.406440Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-04T01:14:32.824Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 59 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "10e84f43-c7b7-9daa-b79a-605d5ed9fdb0" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.35:32984 10.133.0.35:443 10.132.0.10:54812 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T01:14:33.323Z] "DELETE /v1/api-keys/678c02f7-56a4-4c6d-b1bc-ee6e37775317 HTTP/1.1" 200 - via_upstream - "-" 0 321 82 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "eb7cd941-e11c-4669-b3ef-fdfabf6cc408" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.35:32984 10.133.0.35:443 10.132.0.10:54838 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T01:14:38.987Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 49 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "a171a8cd-a489-4a20-b373-8901ca5e0e6f" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.35:443 10.132.0.10:56222 - v1-completions-path 2026-11-04T01:14:41.634992Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:14:41.635029Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:14:41.635044Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:14:41.635049Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T01:14:42.345527Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-04T01:14:41.577Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 57 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "5b4c22b2-f081-44b6-ba07-1e6448f06697" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.35:33034 10.133.0.35:443 10.132.0.10:56234 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T01:14:42.267Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 295 79 - "52.71.36.72,10.132.0.10" "-" "41d2d5a8-a1c4-46cf-907f-4617af2c352a" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:36498 10.133.0.35:443 10.132.0.10:56238 - v1-completions-path 2026-11-04T01:14:48.508639Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:15:00.866015Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:18:05.900Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 136 338 249 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "05beb2d1-3ccd-4d4d-bd1c-b71d8920e735" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:58164 10.133.0.35:443 10.132.0.10:56850 - v1-chat-completions-path 2026-11-04T01:18:13.810103Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:18:13.810138Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:18:13.810151Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T01:18:13.810155Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:18:13.927786Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-11-04T01:18:13.723Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 86 328 86 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "5ed8fe94-a636-4200-ad84-ba1259d183da" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.35:58298 10.133.0.35:443 10.132.0.10:42410 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T01:18:13.866Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 135 2059 66 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "880c02c2-632b-4886-ace4-af9b9d3992a9" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:58164 10.133.0.35:443 10.132.0.10:42426 - v1-chat-completions-path 2026-11-04T01:18:34.256632Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:19:13.589850Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:20:42.701887Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:21:26.329853Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:22:06.081226Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:23:41.961651Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:27:20.105867Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:27:53.549446Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:27:57.911163Z warn xdsproxy upstream terminated with unexpected error rpc error: code = Unavailable desc = error reading from server: read tcp 10.133.0.35:51530->172.31.249.200:15012: read: connection reset by peer id=1 2026-11-04T01:27:58.447746Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-04T01:33:27.898502Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=3 2026-11-04T01:34:26.308710Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:34:50.227261Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:35:37.598040Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:35:53.542841Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:35:55.206833Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:35:58.704133Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:35:58.704172Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:35:58.704192Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:35:58.704198Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T01:35:58.587Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 170 116 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "7ec0be1e-a693-47ab-a443-87919a4548a0" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:35234 10.133.0.35:443 10.132.0.10:37760 - v1-catch-all-path 2026-11-04T01:36:10.263816Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:36:12.049732Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:36:15.184030Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:36:23.011925Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:36:26.343838Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:36:26.343870Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:36:26.343889Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:36:26.343896Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T01:36:26.281Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 77 319 62 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "1ad0d171-d3ce-40cd-b6da-cbb221cf0b50" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.35:44566 10.133.0.35:443 10.132.0.10:43842 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T01:36:27.613Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 59 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "3c98007f-0465-478e-a7fa-121c125326db" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.35:443 10.132.0.10:40732 - v1-completions-path 2026-11-04T01:36:45.024405Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:36:55.215955Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:36:57.005716Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:37:03.668Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 61 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "fe8557f6-f205-4746-8105-aab145cc48cc" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:35234 10.133.0.35:443 10.132.0.10:49278 - v1-completions-path 2026-11-04T01:37:15.800809Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:37:17.517016Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:37:23.035367Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:37:34.377957Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:37:48.068353Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:37:49.695503Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:37:55.138194Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:38:13.303549Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:38:16.362119Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:38:28.376354Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:38:30.011843Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:38:35.828653Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:38:39.129Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 64 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "04ca0689-8a2c-4a66-b552-6cd34169e117" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:54946 10.133.0.35:443 10.132.0.10:46088 - v1-completions-path 2026-11-04T01:38:50.948646Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:39:01.228267Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:39:15.630Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 34 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "5cc02add-3d7d-40c7-a639-cfed3ed639eb" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.35:443 10.132.0.10:53988 - v1-completions-path 2026-11-04T01:39:17.281690Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:39:20.128963Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:41:24.334872Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:41:24.334908Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:41:24.334922Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T01:41:24.334927Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-04T01:41:24.254Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 80 322 80 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "7d85c3f8-1f01-43db-9a6f-d45cecea128d" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.35:34448 10.133.0.35:443 10.132.0.10:51648 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T01:41:35.921536Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:41:38.338277Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:43:05.234031Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:43:20.598745Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:43:22.353346Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:43:28.433628Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:45:41.548508Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:45:41.548545Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:45:41.548557Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T01:45:41.548562Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-04T01:45:41.500Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 48 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "15d5c178-49ff-4432-ba34-51a0b8f4466e" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.35:38514 10.133.0.35:443 10.132.0.10:38204 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T01:45:46.127093Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:46:02.325238Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:46:17.998275Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:46:29.553403Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:46:33.765530Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:46:41.522965Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:46:44.389273Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:46:48.601695Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:46:48.601735Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:46:48.601751Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:46:48.601756Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T01:46:48.398Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 63 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "95b5689f-3ea3-4021-a605-f3ddf888fe5b" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.35:443 10.132.0.10:45970 - v1-completions-path [2026-11-04T01:46:48.539Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 524 61 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "915a22c9-d85c-4e31-9d13-310a2a4e57df" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.35:40154 10.133.0.35:443 10.132.0.10:45984 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T01:47:00.143976Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:49:33.041369Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-04T01:49:33.041393Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-04T01:49:33.041397Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-04T01:49:33.041401Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-11-04T01:49:32.985Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 403 - - "-" 61 103 55 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "e3d59e08-389d-47bc-87a3-908256bc6d72" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.35:443 10.132.0.10:43836 - v1-catch-all-path 2026-11-04T01:49:34.950392Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:49:38.251842Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:49:38.251974Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:49:38.252031Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:49:38.252077Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T01:49:38.386002Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-11-04T01:49:38.178Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 323 74 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "3eec1d01-fa6a-4367-875f-d8c9cd54910a" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "10.132.0.133:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.35:56224 10.133.0.35:443 10.132.0.10:57510 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T01:49:38.309Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 76 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "b4d58772-90fe-4b73-80cd-205fa0ef6bac" "maas.apps.1a199a81a0bfde016d54.prod.konflux-ocp-ci.dev" "10.133.0.33:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:46608 10.133.0.35:443 10.132.0.10:57526 - v1-catch-all-path 2026-11-04T01:49:40.671476Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:49:56.039314Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:49:57.621585Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:50:12.164005Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:50:22.925429Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8