2026-10-30T03:05:11.924528Z info FLAG: --concurrency="0" 2026-10-30T03:05:11.924549Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T03:05:11.924554Z info FLAG: --help="false" 2026-10-30T03:05:11.924557Z info FLAG: --log_as_json="false" 2026-10-30T03:05:11.924561Z info FLAG: --log_caller="" 2026-10-30T03:05:11.924564Z info FLAG: --log_output_level="default:info" 2026-10-30T03:05:11.924567Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T03:05:11.924572Z info FLAG: --log_target="[stdout]" 2026-10-30T03:05:11.924576Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T03:05:11.924579Z info FLAG: --outlierLogPath="" 2026-10-30T03:05:11.924582Z info FLAG: --profiling="true" 2026-10-30T03:05:11.924585Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T03:05:11.924588Z info FLAG: --proxyLogLevel="warning" 2026-10-30T03:05:11.924592Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T03:05:11.924595Z info FLAG: --stsPort="0" 2026-10-30T03:05:11.924597Z info FLAG: --templateFile="" 2026-10-30T03:05:11.924600Z info FLAG: --tokenManagerPlugin="" 2026-10-30T03:05:11.924604Z info FLAG: --vklog="0" 2026-10-30T03:05:11.924607Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T03:05:11.924780Z info Proxy role ips=[10.132.0.176] type=router id=maas-default-gateway-openshift-default-645cf464d7-vszkp.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T03:05:11.924845Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T03:05:11.925879Z info cpu limit detected as 2, setting concurrency 2026-10-30T03:05:11.926114Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T03:05:11.926127Z info JWT policy is third-party-jwt 2026-10-30T03:05:11.926131Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T03:05:11.927836Z info Starting default Istio SDS Server 2026-10-30T03:05:11.927859Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T03:05:11.927883Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T03:05:11.928776Z info Opening status port 15020 2026-10-30T03:05:11.929457Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T03:05:11.931322Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T03:05:11.932416Z info Starting proxy agent 2026-10-30T03:05:11.932448Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T03:05:11.940526Z info sds Starting SDS grpc server 2026-10-30T03:05:11.940545Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T03:05:12.003357Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T03:05:12.068821Z info cache generated new workload certificate resourceName=default latency=139.648398ms ttl=23h59m59.931185961s 2026-10-30T03:05:12.068865Z info cache Root cert has changed, start rotating root cert 2026-10-30T03:05:12.068906Z info cache returned workload trust anchor from cache ttl=23h59m59.931094747s 2026-10-30T03:05:12.191567Z info ads ADS: new connection for node:1 2026-10-30T03:05:12.191632Z info cache returned workload certificate from cache ttl=23h59m59.80836931s 2026-10-30T03:05:12.192869Z info ads ADS: new connection for node:2 2026-10-30T03:05:12.193169Z info cache returned workload trust anchor from cache ttl=23h59m59.806833293s 2026-10-30T03:05:12.312565Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:05:20.662814Z info Readiness succeeded in 8.740820354s 2026-10-30T03:05:20.664134Z info Envoy proxy is ready 2026-10-30T03:06:37.862546Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:19:41.727975Z info ads ADS: new connection for node:3 2026-10-30T03:19:41.747520Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-10-30T03:19:41.747584Z info cache read certificate from file resource=file-root:system 2026-10-30T03:24:15.915904Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:24:40.877695Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:25:01.981457Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:25:41.338196Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:26:18.068141Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:26:22.144218Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:26:22.144261Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:26:22.144277Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:26:22.144283Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T03:26:22.144584Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-10-30T03:26:21.831Z] "POST /anything/e2e-dft-f560b1-35e9e4 HTTP/1.1" 200 - via_upstream - "-" 109 1754 313 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "dfe56a9f-a375-42cb-9158-0ce89708263d" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.176:51610 10.132.0.176:443 10.132.0.13:39542 - llm.external-model-e2e-dft-f560b1-9410f0f45412.0 2026-10-30T03:26:34.258169Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:26:38.582863Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:26:43.183471Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:26:53.984660Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:27:15.060247Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:29:32.837048Z info ads ADS: new connection for node:4 2026-10-30T03:29:32.843765Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T03:29:32.843858Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T03:32:00.096247Z info ads ADS: "" 4 terminated 2026-10-30T03:32:23.978183Z info ads ADS: "" 3 terminated 2026-10-30T03:34:24.315479Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-10-30T03:38:34.838783Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:39:19.766097Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:40:41.980471Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:41:18.239777Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:42:36.951142Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:52:45.677575Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=3 2026-10-30T03:52:52.001968Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:52:52.814364Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:52:52.814415Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:52:52.814439Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:52:52.814447Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T03:52:52.692Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 313 122 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "06ad4002-44c5-4245-b72d-974aac9c4378" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.176:48190 10.132.0.176:443 10.132.0.13:40440 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T03:53:16.238413Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:53:27.411101Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T03:53:27.411136Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T03:53:27.411148Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T03:53:27.411154Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T03:53:27.333Z] "POST /v1/api-keys HTTP/1.1" 400 - via_upstream - "-" 80 94 77 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "09c9a78a-7a33-4b73-9aa7-6605e1d9e08d" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.176:57332 10.132.0.176:443 10.132.0.13:39306 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T03:53:30.904419Z info cache event for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt : REMOVE, pushing to proxy 2026-10-30T03:53:49.962115Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:54:06.178306Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:54:08.622365Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:54:16.153553Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-10-30T03:54:16.070Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 83 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "27163d3b-4410-4cd8-a243-9eddbf1e7b5d" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.176:44672 10.132.0.176:443 10.132.0.13:42762 - v1-catch-all-path 2026-10-30T03:54:19.291925Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:54:35.333859Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:54:37.751238Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:54:52.462554Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:55:03.574099Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:55:05.381159Z info ads ADS: new connection for node:5 2026-10-30T03:55:05.381368Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T03:55:05.381405Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T03:55:07.555889Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:55:49.717481Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:55:52.624372Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T03:55:52.624411Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T03:55:52.624428Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T03:55:52.624435Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T03:55:52.525Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 313 99 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "e82a3ff5-795d-4510-891a-ef4307efdbba" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.176:43364 10.132.0.176:443 10.132.0.13:54884 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T03:55:54.173Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 340 90 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "ee8d12a1-1839-429f-9ac0-eecd504afa89" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.1.67:8000" outbound|54321||e2e-prio-pool-0e6257a1-inference-pool-ip-0cbeffb6.llm.svc.cluster.local 10.132.0.176:47914 10.132.0.176:443 10.132.0.13:54894 - v1-model-routing 2026-10-30T03:56:09.082047Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:56:09.112621Z info ads ADS: "" 5 terminated 2026-10-30T03:56:11.296657Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:56:11.296699Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:56:11.296718Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:56:11.296725Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T03:56:11.439984Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T03:56:11.218Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 90 331 78 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "93aa1765-698b-4adc-9279-8abb98a55125" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.176:35248 10.132.0.176:443 10.132.0.13:47206 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T03:56:14.300161Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:56:19.492442Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:57:55.633014Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:58:13.265772Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:58:31.653502Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-10-30T03:58:31.653606Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:58:31.653633Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:58:31.653653Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:58:31.653660Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T03:58:31.591Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 628 61 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "16b25eca-ec3a-4c77-aaf7-dd246de7da82" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.176:54530 10.132.0.176:443 10.132.0.13:44702 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T03:58:33.718896Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:58:45.701733Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:58:50.604951Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:59:03.319855Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:59:06.999716Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:59:14.398035Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T03:59:14.398089Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T03:59:14.398105Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T03:59:14.398110Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T03:59:14.320Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 335 77 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "bd90c3ba-4f20-4436-bf65-5b9e7401e88c" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.176:41288 10.132.0.176:443 10.132.0.13:38894 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T03:59:15.030246Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:59:15.030287Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:59:15.030303Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T03:59:15.030308Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T03:59:14.979Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 524 51 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "b1691bd9-8cd1-4685-8d0e-75b615729e76" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.176:54530 10.132.0.176:443 10.132.0.13:48664 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T03:59:26.924829Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:00:13.976198Z info ads ADS: new connection for node:6 2026-10-30T04:00:13.980161Z info cache read certificate from file resource=file-root:system 2026-10-30T04:00:15.191721Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:00:15.974342Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:00:19.074340Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:00:19.710109Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:00:19.710159Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:00:19.710185Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:00:19.710193Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T04:00:19.577Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 73 315 132 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "d8948b5a-64dd-4764-b239-8dc0750dbc04" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.176:40826 10.132.0.176:443 10.132.0.13:56202 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T04:00:26.129863Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-10-30T04:00:26.130240Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"900","StreamId":"9761324656647540790"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=25 [2026-10-30T04:00:26.070Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 60 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "e744e4a8-05f3-401f-8742-46963c905889" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.132.0.176:443 10.132.0.13:49100 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-10-30T04:00:28.266421Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:00:28.266451Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:00:28.266461Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T04:00:28.266465Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:00:28.266610Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-10-30T04:00:28.155Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1727 111 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "017a1787-6ab2-4601-a2ea-65c63a74be0e" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.176:59324 10.132.0.176:443 10.132.0.13:49104 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T04:00:30.520517Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:00:30.520551Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:00:30.520562Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:00:30.520567Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T04:00:30.520725Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-10-30T04:00:30.427Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1721 93 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "8296ec65-9a2f-437b-b3bf-8877fd51377a" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.176:59324 10.132.0.176:443 10.132.0.13:49124 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T04:00:34.776769Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:00:34.776807Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:00:34.776819Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:00:34.776823Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T04:00:34.777003Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-10-30T04:00:34.675Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1724 101 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "a4e4ebe3-3ee7-47b1-82d7-38b61aecf94b" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.176:59324 10.132.0.176:443 10.132.0.13:60446 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T04:01:08.925255Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:01:08.925399Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:01:08.925442Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T04:01:08.925468Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:01:08.925764Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T04:01:08.804Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1724 120 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "70e9d181-a86a-487d-91ab-684ff8b97676" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.176:57180 10.132.0.176:443 10.132.0.13:50792 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T04:01:26.019739Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:01:26.019778Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:01:26.019793Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T04:01:26.019800Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:01:26.019988Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T04:01:25.898Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1724 121 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "2294973c-20e7-45c1-ab33-8732edae2f3c" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.176:57180 10.132.0.176:443 10.132.0.13:59030 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T04:01:28.136563Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:01:28.136600Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:01:28.136612Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T04:01:28.136616Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:01:28.136796Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T04:01:28.040Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1724 96 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "ca9bd351-48ff-4983-aedc-a27301e63ccb" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.176:57180 10.132.0.176:443 10.132.0.13:59046 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T04:01:56.575659Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:01:56.681978Z info ads ADS: "" 6 terminated with stream closed 2026-10-30T04:01:57.247169Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:01:59.379575Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:02:01.134118Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T04:02:01.134142Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T04:02:01.134145Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T04:02:01.134149Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-10-30T04:02:01.080Z] "POST /llm/premium-simulated-simulated-premium/v1/completions HTTP/1.1" 403 - - "-" 74 77 53 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "2c1e6361-3075-404b-aed4-78e5d6554660" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "-" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.176:443 10.132.0.13:50492 - v1-completions-path 2026-10-30T04:02:06.344464Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T04:02:10.879Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 75 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "57912e52-9c93-4b94-97c6-952eeb9b393e" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.176:52396 10.132.0.176:443 10.132.0.13:55042 - v1-completions-path 2026-10-30T04:02:23.267845Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:02:27.953638Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:02:43.418813Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:02:59.230257Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T04:03:04.723Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 64 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "afd9d043-ce3e-4f6c-80d7-a4b4aeb1efbc" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.176:55758 10.132.0.176:443 10.132.0.13:40032 - v1-completions-path [2026-10-30T04:03:05.700Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 136 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "60147b8e-974d-485f-8cef-ac9ec35fe9bf" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.176:52396 10.132.0.176:443 10.132.0.13:40034 - v1-completions-path 2026-10-30T04:03:22.636919Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:03:33.083919Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:03:37.588559Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T04:03:41.479Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 63 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "a96b9523-44ae-9865-aa2d-f59ecd8a2388" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.176:55758 10.132.0.176:443 10.132.0.13:59988 - v1-completions-path [2026-10-30T04:03:41.746Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 58 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "db9d280e-f7e0-4201-a834-55aaeff8e277" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.176:55758 10.132.0.176:443 10.132.0.13:60004 - v1-completions-path 2026-10-30T04:03:54.787900Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:03:57.099696Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:04:16.508322Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:04:21.268029Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:04:37.010395Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:04:39.389389Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T04:04:55.923Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 289 81 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "338b8144-8501-468f-8a96-d4af438d3a4b" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.36:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.176:33548 10.132.0.176:443 10.132.0.13:56964 - v1-completions-path 2026-10-30T04:04:57.142905Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:05:07.982608Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:05:10.963499Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:05:16.353682Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:05:16.353717Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:05:16.353732Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:05:16.353737Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T04:05:16.273Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 323 79 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "bad5e94a-46c1-4043-bfdd-ddaa42903634" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.176:49416 10.132.0.176:443 10.132.0.13:55874 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T04:05:29.203971Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:05:32.114203Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T04:05:37.534Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 279 68 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "7aefe3b3-92b3-4e10-9b5d-ef8e2df23274" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.36:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.176:49476 10.132.0.176:443 10.132.0.13:39424 - v1-completions-path 2026-10-30T04:05:44.115143Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:06:03.044300Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:06:13.402184Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:06:33.723762Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:06:37.289634Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:08:20.878162Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:08:24.293885Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:09:54.587782Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:10:09.967895Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:10:12.319454Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T04:10:19.161Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 289 63 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "ec4e5657-6cbc-416e-b0cd-dc7d79c9b789" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.36:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.176:49476 10.132.0.176:443 10.132.0.13:51474 - v1-completions-path 2026-10-30T04:10:31.418359Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:10:43.692829Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:10:47.344417Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T04:10:50.327Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 86 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "ff087ad4-3ba2-47ed-8529-5d66acb0aec1" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.176:52396 10.132.0.176:443 10.132.0.13:43682 - v1-completions-path 2026-10-30T04:11:03.178492Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:11:13.431300Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:11:18.574384Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:11:22.705797Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:11:22.705834Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:11:22.705849Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:11:22.705854Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T04:11:22.628Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 67 309 77 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "8e9a7487-4bdf-48aa-a657-ca2e5b521313" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.1.108:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.176:51408 10.132.0.176:443 10.132.0.13:46266 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T04:11:23.484Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 59 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "5ce9c1e0-39a9-418c-b841-00fd349a253c" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.176:55758 10.132.0.176:443 10.132.0.13:46284 - v1-completions-path [2026-10-30T04:11:23.683Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 94 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "57a82a2e-3032-4721-b857-e233504cebe7" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.176:443 10.132.0.13:46294 - v1-completions-path 2026-10-30T04:11:36.176482Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:11:40.623533Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:11:44.202116Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T04:11:50.701Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 62 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "99ec4c09-e880-90b1-a82d-434f2223facd" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.176:55758 10.132.0.176:443 10.132.0.13:41376 - v1-completions-path [2026-10-30T04:11:51.084Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 65 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "9e2993cc-2afc-49cc-b78d-507ffd7537f9" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.176:52396 10.132.0.176:443 10.132.0.13:41392 - v1-completions-path [2026-10-30T04:11:52.425Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 62 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "7710acc8-e1b0-4c21-8411-dffb49b8c86f" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.176:55758 10.132.0.176:443 10.132.0.13:41402 - v1-completions-path [2026-10-30T04:11:53.905Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 58 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "9e0e3795-11f6-480b-bc90-e404a8cce060" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.176:55758 10.132.0.176:443 10.132.0.13:41424 - v1-completions-path 2026-10-30T04:11:55.250724Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:11:55.250763Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:11:55.250781Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:11:55.250787Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T04:11:55.179Z] "DELETE /v1/api-keys/18973f27-9af9-42e0-bdf7-b8968b2a3508 HTTP/1.1" 200 - via_upstream - "-" 0 365 71 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "346f4529-e85c-4e14-8e3f-2f4416c856ba" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.132.1.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.176:60564 10.132.0.176:443 10.132.0.13:45196 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T04:12:07.245839Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:12:17.569937Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:12:22.209653Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:12:25.665439Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:12:30.585065Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T04:12:35.461Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 68 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "674730c4-f7cf-4796-beef-61b68ca8858b" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.176:55758 10.132.0.176:443 10.132.0.13:45680 - v1-completions-path [2026-10-30T04:12:36.029Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 60 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "79e34c4e-0ab0-4758-b853-08a3c109fc76" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.176:55758 10.132.0.176:443 10.132.0.13:45692 - v1-completions-path 2026-10-30T04:12:36.951554Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:12:36.951591Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:12:36.951607Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:12:36.951613Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T04:12:36.786Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 64 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "9b24e7ba-5df9-4435-81d9-80dadc8cf2da" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.176:443 10.132.0.13:45704 - v1-completions-path [2026-10-30T04:12:36.874Z] "DELETE /v1/api-keys/e192ef00-5471-4b01-8247-329af39afe3d HTTP/1.1" 200 - via_upstream - "-" 0 367 76 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "8ef970e6-ad80-48cf-b110-325b5e543783" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.132.1.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.176:60564 10.132.0.176:443 10.132.0.13:45710 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T04:12:48.920725Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:12:59.258169Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:13:01.552399Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:13:11.309681Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:13:16.399001Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:13:33.137866Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:13:53.278211Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:14:03.521775Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:14:13.853525Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:14:24.307846Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:14:34.567204Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:14:45.008181Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:14:55.600880Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:15:05.887835Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:15:16.192041Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:15:26.556853Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8