2026-11-02T08:31:28.432642Z info FLAG: --concurrency="0" 2026-11-02T08:31:28.432660Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-02T08:31:28.432663Z info FLAG: --help="false" 2026-11-02T08:31:28.432666Z info FLAG: --log_as_json="false" 2026-11-02T08:31:28.432668Z info FLAG: --log_caller="" 2026-11-02T08:31:28.432670Z info FLAG: --log_output_level="default:info" 2026-11-02T08:31:28.432672Z info FLAG: --log_stacktrace_level="default:none" 2026-11-02T08:31:28.432678Z info FLAG: --log_target="[stdout]" 2026-11-02T08:31:28.432680Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-02T08:31:28.432682Z info FLAG: --outlierLogPath="" 2026-11-02T08:31:28.432684Z info FLAG: --profiling="true" 2026-11-02T08:31:28.432686Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-02T08:31:28.432688Z info FLAG: --proxyLogLevel="warning" 2026-11-02T08:31:28.432690Z info FLAG: --serviceCluster="istio-proxy" 2026-11-02T08:31:28.432692Z info FLAG: --stsPort="0" 2026-11-02T08:31:28.432694Z info FLAG: --templateFile="" 2026-11-02T08:31:28.432696Z info FLAG: --tokenManagerPlugin="" 2026-11-02T08:31:28.432698Z info FLAG: --vklog="0" 2026-11-02T08:31:28.432700Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-02T08:31:28.432831Z info Proxy role ips=[10.133.0.183] type=router id=maas-default-gateway-openshift-default-7d64fcddb7-l6g7h.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-02T08:31:28.432874Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-02T08:31:28.433860Z info cpu limit detected as 2, setting concurrency 2026-11-02T08:31:28.434061Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-02T08:31:28.434073Z info JWT policy is third-party-jwt 2026-11-02T08:31:28.434075Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-02T08:31:28.435879Z info Starting default Istio SDS Server 2026-11-02T08:31:28.435898Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-02T08:31:28.435914Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-02T08:31:28.436555Z info Opening status port 15020 2026-11-02T08:31:28.436882Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-02T08:31:28.437553Z info sds Starting SDS grpc server 2026-11-02T08:31:28.437568Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-02T08:31:28.438060Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-02T08:31:28.438802Z info Starting proxy agent 2026-11-02T08:31:28.438832Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-02T08:31:28.502611Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-02T08:31:28.566009Z info cache generated new workload certificate resourceName=default latency=128.366918ms ttl=23h59m59.433993864s 2026-11-02T08:31:28.566039Z info cache Root cert has changed, start rotating root cert 2026-11-02T08:31:28.566072Z info cache returned workload trust anchor from cache ttl=23h59m59.433928846s 2026-11-02T08:31:28.653972Z info ads ADS: new connection for node:1 2026-11-02T08:31:28.654026Z info cache returned workload certificate from cache ttl=23h59m59.345975257s 2026-11-02T08:31:28.654230Z info ads ADS: new connection for node:2 2026-11-02T08:31:28.654295Z info cache returned workload trust anchor from cache ttl=23h59m59.345706292s 2026-11-02T08:31:28.732854Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:31:36.907050Z info Readiness succeeded in 8.476594596s 2026-11-02T08:31:36.907336Z info Envoy proxy is ready 2026-11-02T08:32:02.432233Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T08:32:03.160Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 147 412 1241 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "3de0089d-6310-4d4e-afdb-6104e5f8c3bd" "maas.apps.c6d8fd6add62b0f2acc5.prod.konflux-ocp-ci.dev" "10.132.0.44:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.133.0.183:55706 10.133.0.183:443 10.132.0.19:53550 - v1-chat-completions-path [2026-11-02T08:32:05.935Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 108118 108394 326 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "f9bbee84-38c8-444c-ad5b-b5b58ba35111" "maas.apps.c6d8fd6add62b0f2acc5.prod.konflux-ocp-ci.dev" "10.132.0.44:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.133.0.183:55706 10.133.0.183:443 10.132.0.19:53572 - v1-chat-completions-path 2026-11-02T08:32:20.609367Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:32:25.243150Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:32:35.471958Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:32:39.520931Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 2026-11-02T08:32:40.285418Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-02T08:32:40.286629Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-11-02T08:32:39.426Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 94 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "5a8ce225-dbbd-4957-84f8-4f89c74e9d1a" "maas.apps.c6d8fd6add62b0f2acc5.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.183:41654 10.133.0.183:443 10.132.0.19:50560 - v1-catch-all-path [2026-11-02T08:32:40.220Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 66 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "7b86ed17-c4a9-9016-92bf-65bd7dfc6cac" "maas.apps.c6d8fd6add62b0f2acc5.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.183:41654 10.133.0.183:443 10.132.0.19:50562 - v1-catch-all-path 2026-11-02T08:32:40.638165Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=26 [2026-11-02T08:32:40.571Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 66 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "f8c0ced8-0dc0-430f-b794-3aaa165be0d1" "maas.apps.c6d8fd6add62b0f2acc5.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.183:41664 10.133.0.183:443 10.132.0.19:50566 - v1-catch-all-path 2026-11-02T08:32:48.815870Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:32:58.800854Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:33:15.468814Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:33:27.081123Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:33:30.316154Z info ads ADS: new connection for node:3 2026-11-02T08:33:30.316307Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-02T08:33:30.316337Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-02T08:33:33.700985Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:34:23.345231Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T08:34:33.262Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 341 86 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "4132c8b8-e632-43e9-9fd4-96d411c95ec0" "maas.apps.c6d8fd6add62b0f2acc5.prod.konflux-ocp-ci.dev" "10.132.1.118:8000" outbound|54321||e2e-prio-pool-70e5ebad-inference-pool-ip-07e5532b.llm.svc.cluster.local 10.133.0.183:36194 10.133.0.183:443 10.132.0.19:47786 - v1-model-routing 2026-11-02T08:34:47.305256Z info ads ADS: "" 3 terminated 2026-11-02T08:34:47.605067Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:34:52.259209Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:35:06.540808Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:37:52.359150Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T08:37:52.359182Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T08:37:52.359194Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-02T08:37:52.359199Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 [2026-11-02T08:37:52.301Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 57 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "d9a64957-722e-4797-86e8-4e5a2f790ee6" "maas.apps.c6d8fd6add62b0f2acc5.prod.konflux-ocp-ci.dev" "10.134.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.183:52164 10.133.0.183:443 10.132.0.19:40356 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T08:38:11.735741Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:38:43.244415Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:39:13.713608Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:39:28.150626Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:39:36.434104Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:39:41.141217Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:39:52.207076Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:39:56.588647Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T08:40:02.790Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 52 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "27e623b0-a3d5-496b-ad8f-9c6676a6c372" "maas.apps.c6d8fd6add62b0f2acc5.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.183:443 10.132.0.19:56726 - v1-completions-path 2026-11-02T08:40:16.802262Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:41:14.341185Z info ads ADS: new connection for node:4 2026-11-02T08:41:14.344813Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-02T08:41:14.344852Z info cache read certificate from file resource=file-root:system 2026-11-02T08:41:18.156734Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:41:19.229431Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:41:24.718344Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"190","StreamId":"2161718987116512483"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=25 [2026-11-02T08:41:24.657Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 60 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "0e93cc39-b623-46cd-93b2-6a5a6464b1fc" "maas.apps.c6d8fd6add62b0f2acc5.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.133.0.183:443 10.132.0.19:33728 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-11-02T08:41:31.631753Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:41:31.631786Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:41:31.631799Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T08:41:31.631805Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:41:31.632050Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-02T08:41:31.434Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1718 197 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "41c18386-8d18-4a8e-bb6d-0398669f52bc" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.183:53110 10.133.0.183:443 10.132.0.19:33746 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T08:41:51.025852Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=26 2026-11-02T08:41:51.158886Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T08:41:51.158923Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T08:41:51.158938Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-02T08:41:51.158945Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T08:41:51.159116Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-11-02T08:41:50.963Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1715 195 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "cca6627b-50d2-495f-bc63-a9bc7abe5b26" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.183:50886 10.133.0.183:443 10.132.0.19:36056 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T08:42:03.017809Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:42:03.017839Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:42:03.017848Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T08:42:03.017852Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:42:03.017994Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-02T08:42:02.889Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1718 128 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "214dd440-9e4a-4e79-a375-cd79d2512fa8" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.183:53110 10.133.0.183:443 10.132.0.19:49392 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T08:42:33.477515Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:42:33.477544Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:42:33.477556Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T08:42:33.477560Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T08:42:33.477703Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-02T08:42:33.347Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1719 130 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "3578618a-7721-4534-8969-e886b85dd14e" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.183:53110 10.133.0.183:443 10.132.0.19:40908 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T08:43:02.824959Z info ads ADS: "" 4 terminated with stream closed 2026-11-02T08:43:02.865563Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:43:05.264004Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:43:16.238906Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:43:34.574311Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:43:39.265505Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T08:43:42.215Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 288 78 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "6439f8ea-f03c-481c-b129-4dedaedd6285" "maas.apps.c6d8fd6add62b0f2acc5.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.183:50564 10.133.0.183:443 10.132.0.19:33464 - v1-completions-path [2026-11-02T08:43:44.849Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 61 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "807e3d17-c819-4956-a90b-ac16319da516" "maas.apps.c6d8fd6add62b0f2acc5.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.183:50564 10.133.0.183:443 10.132.0.19:37226 - v1-completions-path [2026-11-02T08:43:45.617Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 58 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "fe9a55f3-6780-412e-82b2-eaffaec9bb47" "maas.apps.c6d8fd6add62b0f2acc5.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.183:443 10.132.0.19:37234 - v1-completions-path 2026-11-02T08:43:59.462567Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:44:18.013352Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T08:44:32.202Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 62 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "b2b78ec0-c0c3-4e9e-9eae-3c23e1d707f3" "maas.apps.c6d8fd6add62b0f2acc5.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.183:50564 10.133.0.183:443 10.132.0.19:35416 - v1-completions-path 2026-11-02T08:44:50.396787Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:45:01.103326Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:45:07.265593Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T08:45:22.023Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 63 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "aa6616da-1657-4cb7-986c-18c4d98f2110" "maas.apps.c6d8fd6add62b0f2acc5.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.183:55158 10.133.0.183:443 10.132.0.19:44606 - v1-completions-path 2026-11-02T08:45:35.740571Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:45:41.914726Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:45:56.120777Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:46:05.809230Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:46:14.916881Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:46:23.473588Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:46:30.949443Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:46:50.455216Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:46:56.978391Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:47:15.296449Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:47:20.836509Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:47:32.188722Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:47:55.809307Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:48:06.691380Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:48:25.993613Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:48:33.374764Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:50:50.917495Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:52:20.667893Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:52:40.283905Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:52:45.239416Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:52:58.623503Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:53:14.625590Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:53:19.579652Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T08:53:25.268Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 58 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "da53a13d-96d8-42e4-922d-5672c0c76afc" "maas.apps.c6d8fd6add62b0f2acc5.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.183:55158 10.133.0.183:443 10.132.0.19:38970 - v1-completions-path 2026-11-02T08:53:39.704023Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:53:50.388784Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:53:56.860951Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T08:54:10.524Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 70 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "b82ae7bd-177f-4c43-9c32-919ca34c50ad" "maas.apps.c6d8fd6add62b0f2acc5.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.183:50564 10.133.0.183:443 10.132.0.19:35102 - v1-completions-path 2026-11-02T08:54:27.061840Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:54:33.533828Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:54:39.486813Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T08:54:50.282Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 64 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "485995d0-5c07-4059-8bc9-28c6bca445db" "maas.apps.c6d8fd6add62b0f2acc5.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.183:55158 10.133.0.183:443 10.132.0.19:53616 - v1-completions-path [2026-11-02T08:54:50.896Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 59 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "606fe689-7514-47ad-800d-6571f4137dd2" "maas.apps.c6d8fd6add62b0f2acc5.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.183:50564 10.133.0.183:443 10.132.0.19:53620 - v1-completions-path [2026-11-02T08:54:57.355Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 67 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "c7a36fde-f69a-41ac-bfad-7081a23e3f7e" "maas.apps.c6d8fd6add62b0f2acc5.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.183:50564 10.133.0.183:443 10.132.0.19:43758 - v1-completions-path 2026-11-02T08:55:12.123523Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:55:22.790616Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:55:29.571058Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:55:35.651157Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:55:43.920788Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T08:55:43.920823Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T08:55:43.920836Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T08:55:43.920841Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-02T08:55:43.792Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 311 128 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "4b22ad27-9170-4125-9924-7edde17bcdcd" "maas.apps.c6d8fd6add62b0f2acc5.prod.konflux-ocp-ci.dev" "10.133.0.191:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.183:49726 10.133.0.183:443 10.132.0.19:35368 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T08:55:47.700679Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T08:55:56.543Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 68 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "89c84f01-e641-4413-9e4f-37926268fcdd" "maas.apps.c6d8fd6add62b0f2acc5.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.183:50564 10.133.0.183:443 10.132.0.19:37298 - v1-completions-path 2026-11-02T08:56:11.253098Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:56:21.970672Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:56:26.996823Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:56:45.350550Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:56:53.376313Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:57:16.019058Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:57:36.526556Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:57:47.578520Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:57:58.692043Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:58:09.539384Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:58:20.276365Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:58:31.073649Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:58:41.822909Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:58:52.630411Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:59:03.662013Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T08:59:10.901847Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-02T08:59:14.176558Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T08:59:30.222Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 92 - "44.212.242.249,10.132.0.19" "python-requests/2.34.2" "316fab50-9a48-439d-b790-852492a220c4" "maas.apps.c6d8fd6add62b0f2acc5.prod.konflux-ocp-ci.dev" "10.134.0.42:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.183:35532 10.133.0.183:443 10.132.0.19:55878 - v1-completions-path