2026-11-04T01:32:33.021760Z info FLAG: --concurrency="0" 2026-11-04T01:32:33.021786Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-04T01:32:33.021791Z info FLAG: --help="false" 2026-11-04T01:32:33.021793Z info FLAG: --log_as_json="false" 2026-11-04T01:32:33.021795Z info FLAG: --log_caller="" 2026-11-04T01:32:33.021797Z info FLAG: --log_output_level="default:info" 2026-11-04T01:32:33.021799Z info FLAG: --log_stacktrace_level="default:none" 2026-11-04T01:32:33.021803Z info FLAG: --log_target="[stdout]" 2026-11-04T01:32:33.021806Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-04T01:32:33.021808Z info FLAG: --outlierLogPath="" 2026-11-04T01:32:33.021810Z info FLAG: --profiling="true" 2026-11-04T01:32:33.021812Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-04T01:32:33.021814Z info FLAG: --proxyLogLevel="warning" 2026-11-04T01:32:33.021817Z info FLAG: --serviceCluster="istio-proxy" 2026-11-04T01:32:33.021819Z info FLAG: --stsPort="0" 2026-11-04T01:32:33.021821Z info FLAG: --templateFile="" 2026-11-04T01:32:33.021823Z info FLAG: --tokenManagerPlugin="" 2026-11-04T01:32:33.021825Z info FLAG: --vklog="0" 2026-11-04T01:32:33.021828Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-04T01:32:33.021966Z info Proxy role ips=[10.132.1.102] type=router id=maas-default-gateway-openshift-default-5d84bc6b95-84hqv.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-04T01:32:33.022008Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-04T01:32:33.023219Z info cpu limit detected as 2, setting concurrency 2026-11-04T01:32:33.023414Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-04T01:32:33.023424Z info JWT policy is third-party-jwt 2026-11-04T01:32:33.023427Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-04T01:32:33.024948Z info Opening status port 15020 2026-11-04T01:32:33.024939Z info Starting default Istio SDS Server 2026-11-04T01:32:33.026110Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-04T01:32:33.026135Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-04T01:32:33.026665Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-04T01:32:33.027927Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-04T01:32:33.028800Z info Starting proxy agent 2026-11-04T01:32:33.028834Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-04T01:32:33.037195Z info sds Starting SDS grpc server 2026-11-04T01:32:33.037213Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-04T01:32:33.092143Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-04T01:32:33.200900Z info cache generated new workload certificate resourceName=default latency=174.081562ms ttl=23h59m59.799104517s 2026-11-04T01:32:33.200938Z info cache Root cert has changed, start rotating root cert 2026-11-04T01:32:33.200983Z info cache returned workload trust anchor from cache ttl=23h59m59.79901794s 2026-11-04T01:32:33.254365Z info ads ADS: new connection for node:1 2026-11-04T01:32:33.254430Z info cache returned workload certificate from cache ttl=23h59m59.745572081s 2026-11-04T01:32:33.254632Z info ads ADS: new connection for node:2 2026-11-04T01:32:33.254706Z info cache returned workload trust anchor from cache ttl=23h59m59.745294992s 2026-11-04T01:32:33.335273Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:32:41.150239Z info Readiness succeeded in 8.131287671s 2026-11-04T01:32:41.150815Z info Envoy proxy is ready 2026-11-04T01:33:08.889596Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:33:10.957367Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:33:10.957404Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:33:10.957417Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T01:33:10.957422Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-04T01:33:10.833Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 61 304 124 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "e382d855-a580-406e-a2bf-bff464b008c0" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.133.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.102:45042 10.132.1.102:443 10.132.0.9:33578 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T01:33:11.469Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 2817 3089 85 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "9969410f-c3a1-4503-b236-c2ff9e5f334e" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.132.1.102:59452 10.132.1.102:443 10.132.0.9:33594 - v1-chat-completions-path [2026-11-04T01:33:11.781Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 108118 108394 314 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "39180126-8fcc-455d-a3c2-79128d8d2eac" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.132.1.102:59452 10.132.1.102:443 10.132.0.9:33600 - v1-chat-completions-path [2026-11-04T01:33:12.496Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 5518 5791 68 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "8646127d-7026-4fbc-973c-363c05d4d404" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.132.1.102:59460 10.132.1.102:443 10.132.0.9:33610 - v1-chat-completions-path 2026-11-04T01:33:27.810843Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:33:32.667187Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:33:42.620193Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:33:45.590093Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-11-04T01:33:45.511Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 78 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "5483fc47-8be7-425e-9eb7-b97c7b1c3d7e" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.1.102:33726 10.132.1.102:443 10.132.0.9:51726 - v1-catch-all-path 2026-11-04T01:33:46.710847Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-04T01:33:46.716346Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-11-04T01:33:46.654Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 61 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "d00b4f4d-fa8f-4001-a32e-ed5992e6456b" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.1.102:33726 10.132.1.102:443 10.132.0.9:51736 - v1-catch-all-path 2026-11-04T01:33:47.485836Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 2026-11-04T01:33:47.883933Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-11-04T01:33:47.420Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 65 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "0bdec70c-65e0-4288-aa20-d38e791dd016" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.1.102:33726 10.132.1.102:443 10.132.0.9:51750 - v1-catch-all-path [2026-11-04T01:33:47.793Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 90 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "a6aede15-65cc-4dd3-b0ce-1fb8ce2931fe" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.1.102:33728 10.132.1.102:443 10.132.0.9:51766 - v1-catch-all-path 2026-11-04T01:33:57.125136Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:34:06.878502Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:34:23.155072Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:34:35.095896Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:34:37.038094Z info ads ADS: new connection for node:3 2026-11-04T01:34:37.038283Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-04T01:34:37.038305Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-04T01:34:40.521593Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:35:21.360055Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:35:45.741800Z info ads ADS: "" 3 terminated with stream closed 2026-11-04T01:35:46.132528Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:35:51.474815Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:36:08.202036Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:38:42.486705Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:39:10.843759Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:39:30.478125Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:39:44.474308Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:39:47.645819Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-11-04T01:39:47.651955Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:39:47.651989Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:39:47.652002Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:39:47.652008Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T01:39:47.917418Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:39:47.917457Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:39:47.917488Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T01:39:47.917493Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-04T01:39:47.538Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 84 326 113 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "5a6f5fed-bb8b-4d87-8a9c-c4662a3228ee" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.133.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.102:58710 10.132.1.102:443 10.132.0.9:35434 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T01:39:47.865Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 51 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "ab7ad761-a539-4767-85ae-a3b2bb2e3175" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.133.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.102:58710 10.132.1.102:443 10.132.0.9:35438 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T01:39:51.533483Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:40:02.444756Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:40:07.539396Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:40:25.676260Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:40:28.674Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 78 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "bf5e4d98-3bd0-4ef5-8bd2-ec39ef5bd093" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.132.0.51:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.1.102:58722 10.132.1.102:443 10.132.0.9:57374 - v1-completions-path 2026-11-04T01:41:13.689267Z info ads ADS: new connection for node:4 2026-11-04T01:41:13.693326Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-04T01:41:13.693374Z info cache read certificate from file resource=file-root:system 2026-11-04T01:41:17.310336Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:41:18.849909Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:41:38.433757Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:41:38.433796Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:41:38.433812Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T01:41:38.433818Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:41:38.434022Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-04T01:41:38.230Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1532 204 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "8e34b43e-ca62-47cf-8ad4-73e5d76d1718" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.1.102:36196 10.132.1.102:443 10.132.0.9:49674 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-04T01:41:43.232216Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:41:43.232256Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:41:43.232270Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:41:43.232276Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T01:41:43.232504Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-04T01:41:43.040Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1534 191 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "107fc518-d0d6-4688-8157-f3c531866b71" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.1.102:37812 10.132.1.102:443 10.132.0.9:49502 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-04T01:41:55.080823Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:41:55.080853Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:41:55.080863Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T01:41:55.080867Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:41:55.081022Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-04T01:41:54.950Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1533 130 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "b1973fc4-33d2-4f3f-91db-cc3dc0e6aa05" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.1.102:37812 10.132.1.102:443 10.132.0.9:60754 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-04T01:42:55.585489Z info ads ADS: "" 4 terminated 2026-11-04T01:42:57.099048Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:42:58.370074Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:43:09.281220Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:43:13.336Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 78 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "990f4a7f-f51d-4c2e-a87c-54d9423d8ac3" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.102:34006 10.132.1.102:443 10.132.0.9:55800 - v1-completions-path 2026-11-04T01:43:27.786120Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:43:32.683084Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:43:35.454Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 300 66 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "b0976dbc-dee5-4a94-815c-2ab801e3290e" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.102:34006 10.132.1.102:443 10.132.0.9:49506 - v1-completions-path [2026-11-04T01:43:37.189Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 64 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "bde53712-2dde-49a0-9c82-383063382dfd" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.102:37680 10.132.1.102:443 10.132.0.9:49536 - v1-completions-path [2026-11-04T01:43:37.480Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 288 62 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "7339a6ac-7eac-4ef9-87a0-def721b7a467" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.102:37680 10.132.1.102:443 10.132.0.9:49544 - v1-completions-path [2026-11-04T01:43:37.751Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 59 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "d19f3d5d-a955-4a70-84ab-165008c5d1af" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.102:34006 10.132.1.102:443 10.132.0.9:49550 - v1-completions-path [2026-11-04T01:43:37.998Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 299 72 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "4e921eaf-970c-4a97-8e25-feb47d0d4242" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.102:34006 10.132.1.102:443 10.132.0.9:49552 - v1-completions-path 2026-11-04T01:43:57.273705Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:44:10.421076Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:44:16.926457Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:44:16.926523Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:44:16.926537Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:44:16.926543Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-04T01:44:16.810Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 323 116 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "d591e073-2059-40c9-95c8-e9f1a41a4d94" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.133.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.102:47382 10.132.1.102:443 10.132.0.9:48964 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T01:44:19.368Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 57 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "87720756-8cf0-463f-b706-954aaf0f2422" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.1.102:443 10.132.0.9:48970 - v1-completions-path [2026-11-04T01:44:23.369Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 64 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "4de094ad-b84e-473f-a2e3-985fbc5ad4c4" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.102:37680 10.132.1.102:443 10.132.0.9:39674 - v1-completions-path [2026-11-04T01:44:24.505Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 297 60 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "34115f80-54cc-44ac-a2bd-ff649c17ab19" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.102:37680 10.132.1.102:443 10.132.0.9:39688 - v1-completions-path 2026-11-04T01:44:41.605178Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:44:52.326917Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:44:57.392633Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:45:11.991Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 61 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "b64d3d5d-53ca-4079-8d3a-908121f205de" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.102:37680 10.132.1.102:443 10.132.0.9:60396 - v1-completions-path 2026-11-04T01:45:26.958727Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:45:31.934023Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:45:35.198Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 74 288 75 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "b96165bc-8d31-4a97-8beb-2e359467af80" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.132.0.52:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.132.1.102:44010 10.132.1.102:443 10.132.0.9:57698 - v1-completions-path 2026-11-04T01:45:46.058203Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:45:55.471162Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:46:03.588498Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:46:12.938163Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:46:22.662140Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:46:41.209270Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:47:12.731313Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:47:36.129694Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:47:46.845693Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:52:47.782356Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:53:05.030291Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:53:09.621087Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:53:22.313723Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:53:39.980049Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:53:46.529726Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:53:50.653Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 61 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "ef75b8ad-38fa-48a1-be4f-37427a673f1b" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.102:37680 10.132.1.102:443 10.132.0.9:33250 - v1-completions-path 2026-11-04T01:54:05.946488Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:54:16.631312Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:54:22.838079Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:54:34.046376Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:54:34.046411Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:54:34.046424Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:54:34.046430Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-04T01:54:33.926Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 67 309 120 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "2fcac20f-c485-43bf-a3c8-406684be599b" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.133.0.192:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.102:53042 10.132.1.102:443 10.132.0.9:56058 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T01:54:50.740676Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:54:56.906056Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:55:02.762992Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:55:11.813Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 70 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "95f01b52-a090-4ec7-9c84-72773fb07983" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.102:34006 10.132.1.102:443 10.132.0.9:49812 - v1-completions-path [2026-11-04T01:55:15.085Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 64 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "8db24443-c975-4e75-a98d-ae1dd8ec4b2e" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.102:34006 10.132.1.102:443 10.132.0.9:49828 - v1-completions-path 2026-11-04T01:55:34.464747Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:55:45.270385Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:55:51.691555Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:55:59.249887Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:56:10.445764Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:56:33.838979Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:56:44.642158Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:56:49.423704Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:57:07.810716Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:57:13.894057Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:57:35.962057Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:57:57.853677Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:58:18.158549Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:58:30.659003Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:58:50.679421Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:59:01.602050Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:59:12.295401Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:59:23.154642Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:59:33.751405Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:59:44.502Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 289 75 - "44.212.242.249,10.132.0.9" "python-requests/2.34.2" "a5f7db79-c751-4fe3-8f85-c8ab48d5486b" "maas.apps.34861718878ef819d16d.prod.konflux-ocp-ci.dev" "10.132.0.51:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.1.102:58722 10.132.1.102:443 10.132.0.9:37812 - v1-completions-path