2026-10-30T03:47:54.126214Z info FLAG: --concurrency="0" 2026-10-30T03:47:54.126233Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T03:47:54.126237Z info FLAG: --help="false" 2026-10-30T03:47:54.126240Z info FLAG: --log_as_json="false" 2026-10-30T03:47:54.126242Z info FLAG: --log_caller="" 2026-10-30T03:47:54.126244Z info FLAG: --log_output_level="default:info" 2026-10-30T03:47:54.126246Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T03:47:54.126251Z info FLAG: --log_target="[stdout]" 2026-10-30T03:47:54.126253Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T03:47:54.126255Z info FLAG: --outlierLogPath="" 2026-10-30T03:47:54.126258Z info FLAG: --profiling="true" 2026-10-30T03:47:54.126260Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T03:47:54.126262Z info FLAG: --proxyLogLevel="warning" 2026-10-30T03:47:54.126264Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T03:47:54.126266Z info FLAG: --stsPort="0" 2026-10-30T03:47:54.126268Z info FLAG: --templateFile="" 2026-10-30T03:47:54.126270Z info FLAG: --tokenManagerPlugin="" 2026-10-30T03:47:54.126272Z info FLAG: --vklog="0" 2026-10-30T03:47:54.126275Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T03:47:54.126430Z info Proxy role ips=[10.132.0.147] type=router id=maas-default-gateway-openshift-default-77bfb58f8c-lkn75.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T03:47:54.126478Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T03:47:54.127581Z info cpu limit detected as 2, setting concurrency 2026-10-30T03:47:54.128184Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T03:47:54.128196Z info JWT policy is third-party-jwt 2026-10-30T03:47:54.128199Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T03:47:54.129605Z info Starting default Istio SDS Server 2026-10-30T03:47:54.129632Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T03:47:54.129651Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T03:47:54.129972Z info Opening status port 15020 2026-10-30T03:47:54.131023Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T03:47:54.132443Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T03:47:54.133279Z info Starting proxy agent 2026-10-30T03:47:54.133309Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T03:47:54.142529Z info sds Starting SDS grpc server 2026-10-30T03:47:54.142551Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T03:47:54.201077Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T03:47:54.279396Z info cache generated new workload certificate resourceName=default latency=148.555333ms ttl=23h59m59.720610378s 2026-10-30T03:47:54.279445Z info cache Root cert has changed, start rotating root cert 2026-10-30T03:47:54.279497Z info cache returned workload trust anchor from cache ttl=23h59m59.720504344s 2026-10-30T03:47:54.406881Z info ads ADS: new connection for node:1 2026-10-30T03:47:54.410679Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-10-30T03:47:54.410724Z info cache read certificate from file resource=file-root:system 2026-10-30T03:47:54.438541Z info ads ADS: new connection for node:2 2026-10-30T03:47:54.438615Z info cache returned workload certificate from cache ttl=23h59m59.561386388s 2026-10-30T03:47:54.439660Z info ads ADS: new connection for node:3 2026-10-30T03:47:54.439750Z info cache returned workload trust anchor from cache ttl=23h59m59.560251538s 2026-10-30T03:47:54.540032Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T03:47:57.749589Z error failed scraping envoy metrics: error scraping http://localhost:15090/stats/prometheus: Get "http://localhost:15090/stats/prometheus": dial tcp [::1]:15090: connect: connection refused 2026-10-30T03:48:03.046688Z info Readiness succeeded in 8.922832198s 2026-10-30T03:48:03.047138Z info Envoy proxy is ready 2026-10-30T03:48:16.990660Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T03:49:43.564367Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T03:50:10.068723Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T03:50:20.028243Z info ads ADS: new connection for node:4 2026-10-30T03:50:20.028458Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T03:50:20.028491Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T03:50:26.265075Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T03:50:45.672857Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T03:52:49.041661Z info ads ADS: "" 4 terminated with stream closed 2026-10-30T03:53:14.831027Z info ads ADS: "" 1 terminated with stream closed 2026-10-30T03:57:19.692750Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T03:58:41.705895Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:00:41.439889Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:01:11.034573Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:02:45.335453Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:11:58.921788Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:12:20.471436Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:12:52.990776Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-10-30T04:12:54.915669Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:13:10.691128Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:13:13.207163Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:13:15.901501Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=26 [2026-10-30T04:13:15.807Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 93 - "34.228.250.194,10.133.0.13" "python-requests/2.34.2" "eb9575eb-44be-470b-bcb4-205fb93ef10d" "maas.apps.1ef8b236caec72cd892e.prod.konflux-ocp-ci.dev" "10.133.0.51:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.147:59338 10.132.0.147:443 10.133.0.13:48400 - v1-catch-all-path 2026-10-30T04:13:19.734275Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:13:22.930551Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=26 [2026-10-30T04:13:22.868Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 61 - "34.228.250.194,10.133.0.13" "python-requests/2.34.2" "da5aa35b-ae7a-42e0-8129-d87859b90ecb" "maas.apps.1ef8b236caec72cd892e.prod.konflux-ocp-ci.dev" "10.133.0.51:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.147:59338 10.132.0.147:443 10.133.0.13:60640 - v1-catch-all-path 2026-10-30T04:13:27.243916Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:13:36.523558Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:13:51.201314Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:14:02.163276Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:14:04.009744Z info ads ADS: new connection for node:5 2026-10-30T04:14:04.009968Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T04:14:06.042257Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:14:46.807305Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:14:47.687811Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T04:14:47.687851Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T04:14:47.687867Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-10-30T04:14:47.687872Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 [2026-10-30T04:14:47.577Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 110 - "34.228.250.194,10.133.0.13" "python-requests/2.34.2" "6dca7405-6194-4aca-b238-565e5b037689" "maas.apps.1ef8b236caec72cd892e.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.147:58020 10.132.0.147:443 10.133.0.13:48744 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T04:15:05.691870Z info ads ADS: "" 5 terminated 2026-10-30T04:15:05.740717Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:15:09.349782Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:15:17.464404Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:16:38.674267Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T04:16:38.674309Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T04:16:38.674325Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T04:16:38.674330Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-10-30T04:16:38.674337Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=26 [2026-10-30T04:16:38.600Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 73 - "34.228.250.194,10.133.0.13" "python-requests/2.34.2" "3a7d3e80-ce7c-4aa2-ac05-c601ca842d93" "maas.apps.1ef8b236caec72cd892e.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.147:54966 10.132.0.147:443 10.133.0.13:50396 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T04:16:45.031420Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:17:01.349050Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:17:17.502996Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:17:29.307990Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:17:33.374535Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:17:44.396652Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:17:47.982613Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:18:03.000192Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-10-30T04:18:34.404Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 284 87 - "34.228.250.194,10.133.0.13" "python-requests/2.34.2" "8879666f-b457-43a9-84d8-42237f811fa3" "maas.apps.1ef8b236caec72cd892e.prod.konflux-ocp-ci.dev" "10.133.0.53:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.147:44274 10.132.0.147:443 10.133.0.13:43494 - v1-completions-path 2026-10-30T04:18:50.201730Z info ads ADS: new connection for node:6 2026-10-30T04:18:50.207146Z info cache read certificate from file resource=file-root:system 2026-10-30T04:18:51.267116Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:18:52.915385Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:19:55.290471Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:19:55.290529Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:19:55.290544Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:19:55.290551Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T04:19:55.290797Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-10-30T04:19:55.176Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1722 114 - "34.228.250.194,10.133.0.13" "python-requests/2.34.2" "d3fd84b3-1897-485b-b118-b372214f0236" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.147:35162 10.132.0.147:443 10.133.0.13:46120 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T04:20:08.053319Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T04:20:08.053357Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T04:20:08.053368Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-10-30T04:20:08.053373Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T04:20:08.053568Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-10-30T04:20:07.942Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1716 110 - "34.228.250.194,10.133.0.13" "python-requests/2.34.2" "bac5f3c9-7a31-4760-abfc-1c870d6b09b7" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.147:37756 10.132.0.147:443 10.133.0.13:58556 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T04:20:40.583144Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:20:40.664990Z info ads ADS: "" 6 terminated with stream closed 2026-10-30T04:20:42.056946Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:20:48.765764Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:21:04.508694Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:21:09.313958Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:21:26.202876Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:21:40.445804Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-10-30T04:21:46.280Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 76 - "34.228.250.194,10.133.0.13" "python-requests/2.34.2" "b35f76df-3a1c-467c-bf91-a3c9a6a0c99c" "maas.apps.1ef8b236caec72cd892e.prod.konflux-ocp-ci.dev" "10.133.0.52:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.147:34794 10.132.0.147:443 10.133.0.13:60394 - v1-completions-path 2026-10-30T04:22:03.906279Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:22:14.126046Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:22:18.926634Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-10-30T04:22:23.041Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 75 - "34.228.250.194,10.133.0.13" "python-requests/2.34.2" "bfe688f5-e599-45e6-a7ff-e5d686b1ed81" "maas.apps.1ef8b236caec72cd892e.prod.konflux-ocp-ci.dev" "10.133.0.52:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.147:48848 10.132.0.147:443 10.133.0.13:56226 - v1-completions-path 2026-10-30T04:22:23.885110Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-10-30T04:22:23.494Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 61 - "34.228.250.194,10.133.0.13" "python-requests/2.34.2" "de901c6c-7b72-4dcd-9b70-76df68e69c87" "maas.apps.1ef8b236caec72cd892e.prod.konflux-ocp-ci.dev" "10.133.0.52:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.147:34794 10.132.0.147:443 10.133.0.13:56230 - v1-completions-path [2026-10-30T04:22:23.746Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 64 - "34.228.250.194,10.133.0.13" "python-requests/2.34.2" "7057afb4-7386-4303-b6fe-f096c38969e8" "maas.apps.1ef8b236caec72cd892e.prod.konflux-ocp-ci.dev" "10.133.0.52:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.147:34794 10.132.0.147:443 10.133.0.13:56244 - v1-completions-path [2026-10-30T04:22:23.830Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 58 - "34.228.250.194,10.133.0.13" "python-requests/2.34.2" "8a7ba1f8-3bcf-463a-9d9a-d1f7e205f37f" "maas.apps.1ef8b236caec72cd892e.prod.konflux-ocp-ci.dev" "10.133.0.52:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.147:48848 10.132.0.147:443 10.133.0.13:56250 - v1-completions-path 2026-10-30T04:22:36.018729Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:22:38.408665Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:22:45.737333Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:22:57.548826Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:23:02.868150Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:23:10.987192Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:23:16.358388Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-10-30T04:23:22.658Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 290 69 - "34.228.250.194,10.133.0.13" "python-requests/2.34.2" "c8da955b-50f6-4f98-98e0-7a16abacebf3" "maas.apps.1ef8b236caec72cd892e.prod.konflux-ocp-ci.dev" "10.133.0.53:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.147:57892 10.132.0.147:443 10.133.0.13:40296 - v1-completions-path 2026-10-30T04:23:34.666351Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:23:37.740672Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:23:53.255179Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:23:56.184274Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:24:04.247885Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:24:22.215838Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:24:32.583827Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:24:49.946627Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:24:53.460037Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:26:24.595337Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:26:27.995715Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:27:53.207722Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:28:08.522365Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:28:10.915997Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:28:18.104097Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:28:38.340508Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:28:41.402052Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-10-30T04:28:41.845Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 111 - "34.228.250.194,10.133.0.13" "python-requests/2.34.2" "1fda990f-443c-4220-be6c-875ded527adb" "maas.apps.1ef8b236caec72cd892e.prod.konflux-ocp-ci.dev" "10.133.0.52:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.147:34794 10.132.0.147:443 10.133.0.13:35310 - v1-completions-path [2026-10-30T04:28:44.268Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 73 - "34.228.250.194,10.133.0.13" "python-requests/2.34.2" "1f1c0116-33dd-406c-9251-c34c82ca3d10" "maas.apps.1ef8b236caec72cd892e.prod.konflux-ocp-ci.dev" "10.133.0.52:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.147:34794 10.132.0.147:443 10.133.0.13:35330 - v1-completions-path 2026-10-30T04:28:57.013968Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:29:07.353018Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:29:12.483091Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:29:17.975279Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T04:29:17.975318Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T04:29:17.975334Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-10-30T04:29:17.975339Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 [2026-10-30T04:29:17.897Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 67 309 77 - "34.228.250.194,10.133.0.13" "python-requests/2.34.2" "02e771b7-0506-4c94-a663-a109e77e3b15" "maas.apps.1ef8b236caec72cd892e.prod.konflux-ocp-ci.dev" "10.133.1.124:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.147:53640 10.132.0.147:443 10.133.0.13:50810 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T04:29:31.202812Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:29:35.866322Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:29:38.734788Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:29:45.190608Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T04:29:45.190647Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T04:29:45.190664Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T04:29:45.190670Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-10-30T04:29:45.115Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 68 311 74 - "34.228.250.194,10.133.0.13" "python-requests/2.34.2" "c07c8d52-0671-4e0e-a55d-6d6114cb2f65" "maas.apps.1ef8b236caec72cd892e.prod.konflux-ocp-ci.dev" "10.133.1.124:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.147:53640 10.132.0.147:443 10.133.0.13:44210 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T04:29:46.375Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 63 - "34.228.250.194,10.133.0.13" "python-requests/2.34.2" "a132de64-18c2-40c5-9bd2-11bced8db1cd" "maas.apps.1ef8b236caec72cd892e.prod.konflux-ocp-ci.dev" "10.133.0.52:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.147:48848 10.132.0.147:443 10.133.0.13:44232 - v1-completions-path [2026-10-30T04:29:46.745Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 59 - "34.228.250.194,10.133.0.13" "python-requests/2.34.2" "78a94562-6aa2-4d0d-b395-0689bb2feb9d" "maas.apps.1ef8b236caec72cd892e.prod.konflux-ocp-ci.dev" "10.133.0.52:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.147:48848 10.132.0.147:443 10.133.0.13:44244 - v1-completions-path 2026-10-30T04:30:01.840853Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:30:12.206271Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:30:14.708064Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:30:17.641966Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:30:22.845914Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:30:41.128714Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:30:51.478122Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:30:53.935764Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:31:03.649477Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:31:08.546495Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:31:25.371474Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:31:45.244238Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:31:55.538084Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:32:05.882885Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:32:16.307583Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:32:26.594222Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:32:36.926387Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:32:47.292909Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:32:57.690094Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:33:07.971171Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-10-30T04:33:18.242868Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-10-30T04:33:25.074Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 84 - "34.228.250.194,10.133.0.13" "python-requests/2.34.2" "2d5c22d6-0ace-460e-840b-1846d21a3862" "maas.apps.1ef8b236caec72cd892e.prod.konflux-ocp-ci.dev" "10.133.0.53:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.147:44274 10.132.0.147:443 10.133.0.13:46618 - v1-completions-path