2026-11-03T22:11:28.749283Z info FLAG: --concurrency="0" 2026-11-03T22:11:28.749303Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T22:11:28.749307Z info FLAG: --help="false" 2026-11-03T22:11:28.749309Z info FLAG: --log_as_json="false" 2026-11-03T22:11:28.749311Z info FLAG: --log_caller="" 2026-11-03T22:11:28.749313Z info FLAG: --log_output_level="default:info" 2026-11-03T22:11:28.749315Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T22:11:28.749319Z info FLAG: --log_target="[stdout]" 2026-11-03T22:11:28.749321Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T22:11:28.749323Z info FLAG: --outlierLogPath="" 2026-11-03T22:11:28.749325Z info FLAG: --profiling="true" 2026-11-03T22:11:28.749327Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T22:11:28.749329Z info FLAG: --proxyLogLevel="warning" 2026-11-03T22:11:28.749331Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T22:11:28.749333Z info FLAG: --stsPort="0" 2026-11-03T22:11:28.749335Z info FLAG: --templateFile="" 2026-11-03T22:11:28.749337Z info FLAG: --tokenManagerPlugin="" 2026-11-03T22:11:28.749340Z info FLAG: --vklog="0" 2026-11-03T22:11:28.749342Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T22:11:28.749475Z info Proxy role ips=[10.133.1.94] type=router id=maas-default-gateway-openshift-default-8b9c49cd7-7lrp6.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T22:11:28.749519Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T22:11:28.750531Z info cpu limit detected as 2, setting concurrency 2026-11-03T22:11:28.751120Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T22:11:28.751132Z info JWT policy is third-party-jwt 2026-11-03T22:11:28.751135Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T22:11:28.752783Z info Starting default Istio SDS Server 2026-11-03T22:11:28.752804Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T22:11:28.752822Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T22:11:28.753269Z info Opening status port 15020 2026-11-03T22:11:28.753710Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T22:11:28.754453Z info sds Starting SDS grpc server 2026-11-03T22:11:28.754469Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T22:11:28.755012Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T22:11:28.755770Z info Starting proxy agent 2026-11-03T22:11:28.755800Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T22:11:28.816930Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T22:11:28.917921Z info cache generated new workload certificate resourceName=default latency=163.326918ms ttl=23h59m59.082081395s 2026-11-03T22:11:28.917954Z info cache Root cert has changed, start rotating root cert 2026-11-03T22:11:28.917983Z info cache returned workload trust anchor from cache ttl=23h59m59.082017815s 2026-11-03T22:11:28.977794Z info ads ADS: new connection for node:1 2026-11-03T22:11:28.977848Z info cache returned workload certificate from cache ttl=23h59m59.022153555s 2026-11-03T22:11:28.978361Z info ads ADS: new connection for node:2 2026-11-03T22:11:28.978447Z info cache returned workload trust anchor from cache ttl=23h59m59.021554938s 2026-11-03T22:11:29.061540Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:11:37.054804Z info Readiness succeeded in 8.307699063s 2026-11-03T22:11:37.055394Z info Envoy proxy is ready 2026-11-03T22:11:39.456211Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:12:15.985276Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T22:12:21.035Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 144 409 84 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "5958f7e9-48f4-4330-b220-7f07d4f6f19e" "maas.apps.4ddbac784716b2f7da33.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.133.1.94:33358 10.133.1.94:443 10.132.0.21:48580 - v1-chat-completions-path [2026-11-03T22:12:21.320Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 140 405 64 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "3c37510f-4066-4873-84bf-dd3000f8db57" "maas.apps.4ddbac784716b2f7da33.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.133.1.94:33368 10.133.1.94:443 10.132.0.21:48590 - v1-chat-completions-path 2026-11-03T22:12:35.326100Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:12:40.303492Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:12:48.814956Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:13:03.158338Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:13:12.264854Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:13:29.275358Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:13:40.992053Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:13:44.590160Z info ads ADS: new connection for node:3 2026-11-03T22:13:44.590331Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T22:13:44.590361Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T22:13:47.995801Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:14:26.895227Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:14:51.967853Z info ads ADS: "" 3 terminated 2026-11-03T22:14:52.031769Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:14:56.424899Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:15:13.153810Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:18:00.370510Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:18:30.145203Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:19:00.551883Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:19:16.126512Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:19:31.224580Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:19:43.710136Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:19:48.851862Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:19:55.971896Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:19:55.971933Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:19:55.971947Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:19:55.971952Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T22:19:55.916Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 524 55 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "85c6db52-aeca-47e3-a289-6e5b52acc7df" "maas.apps.4ddbac784716b2f7da33.prod.konflux-ocp-ci.dev" "10.134.0.28:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.1.94:37414 10.133.1.94:443 10.132.0.21:42474 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T22:20:09.704498Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T22:20:25.948Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 290 87 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "a5eb0700-00e4-4a0c-b052-1e0803d54356" "maas.apps.4ddbac784716b2f7da33.prod.konflux-ocp-ci.dev" "10.134.0.37:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.1.94:36424 10.133.1.94:443 10.132.0.21:57650 - v1-completions-path 2026-11-03T22:20:57.379569Z info ads ADS: new connection for node:4 2026-11-03T22:20:57.386757Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-03T22:20:57.386812Z info cache read certificate from file resource=file-root:system 2026-11-03T22:21:01.211199Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:21:02.381945Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:21:07.974738Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"177","StreamId":"4292398803540255688"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=25 [2026-11-03T22:21:07.911Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 63 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "b3c192d7-b499-4bb1-99b2-b137fd86cfae" "maas.apps.4ddbac784716b2f7da33.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.133.1.94:443 10.132.0.21:60186 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-11-03T22:21:26.929904Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:21:26.929937Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:21:26.929948Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T22:21:26.929953Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:21:26.930098Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T22:21:26.738Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1538 191 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "9d1c7b66-cba6-4921-9d35-c6011ffe3a88" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.1.94:59302 10.133.1.94:443 10.132.0.21:39748 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T22:21:38.837777Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-03T22:21:38.838106Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:21:38.838140Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:21:38.838155Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:21:38.838161Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T22:21:38.838349Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T22:21:38.651Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1537 186 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "d015b223-2fed-4c72-a577-295b3592d953" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.1.94:50854 10.133.1.94:443 10.132.0.21:52098 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T22:22:07.137439Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-11-03T22:22:07.137881Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:22:07.137905Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:22:07.137915Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:22:07.137919Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T22:22:07.138066Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T22:22:07.015Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1538 122 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "77501af1-fb52-4369-919a-4bc48ccdbdb2" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.1.94:59302 10.133.1.94:443 10.132.0.21:33590 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T22:22:21.386835Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:22:21.386867Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:22:21.386877Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:22:21.386881Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T22:22:21.387024Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T22:22:21.261Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1539 125 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "c017601c-46b6-4b13-a610-2997e0ea2b16" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.1.94:59302 10.133.1.94:443 10.132.0.21:45664 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T22:22:57.776890Z info ads ADS: "" 4 terminated with stream closed 2026-11-03T22:22:59.420128Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:23:00.469508Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:23:11.173120Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:23:29.663649Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:23:34.311566Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T22:23:38.035Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 298 72 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "8063ece1-9fc0-4e0f-a39a-630c25ad6d47" "maas.apps.4ddbac784716b2f7da33.prod.konflux-ocp-ci.dev" "10.134.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.94:37198 10.133.1.94:443 10.132.0.21:49138 - v1-completions-path [2026-11-03T22:23:41.178Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 54 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "34c80377-99b2-4d8d-831e-3779f29d31d5" "maas.apps.4ddbac784716b2f7da33.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.1.94:443 10.132.0.21:49156 - v1-completions-path 2026-11-03T22:23:58.860156Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:24:11.989567Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T22:24:18.983Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 302 63 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "f6e6fac6-c6eb-425b-b906-cf12f3e7ba3a" "maas.apps.4ddbac784716b2f7da33.prod.konflux-ocp-ci.dev" "10.134.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.94:37198 10.133.1.94:443 10.132.0.21:52104 - v1-completions-path [2026-11-03T22:24:20.659Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 57 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "b549227c-b399-495b-82d8-df465842f77f" "maas.apps.4ddbac784716b2f7da33.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.1.94:443 10.132.0.21:52120 - v1-completions-path [2026-11-03T22:24:21.938Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 53 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "005383b1-327e-41dd-8334-d0647698443e" "maas.apps.4ddbac784716b2f7da33.prod.konflux-ocp-ci.dev" "10.134.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.94:37198 10.133.1.94:443 10.132.0.21:52134 - v1-completions-path 2026-11-03T22:24:42.634698Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:24:53.244866Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:24:59.718840Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:25:10.193154Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:25:10.193190Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:25:10.193202Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T22:25:10.193207Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T22:25:10.082Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 317 110 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "364f75bb-f18c-4726-8715-a4ea0046e4a9" "maas.apps.4ddbac784716b2f7da33.prod.konflux-ocp-ci.dev" "10.134.0.28:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.1.94:46846 10.133.1.94:443 10.132.0.21:46938 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T22:25:14.300Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 61 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "d68a3ba8-9429-4109-9ab5-475ff80d2710" "maas.apps.4ddbac784716b2f7da33.prod.konflux-ocp-ci.dev" "10.134.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.94:39114 10.133.1.94:443 10.132.0.21:46952 - v1-completions-path 2026-11-03T22:25:28.086357Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:25:32.925654Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:25:46.517309Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:25:55.715447Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:26:06.339399Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:26:12.810320Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:26:27.757918Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:26:39.826555Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:26:44.821180Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:27:03.350049Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:27:08.997182Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:27:20.374306Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:27:32.103714Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:27:32.103749Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:27:32.103762Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:27:32.103767Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T22:27:32.417107Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:27:32.417140Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:27:32.417155Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T22:27:32.417160Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T22:27:31.994Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 67 310 109 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "481be442-041b-4358-82b5-2853c581b251" "maas.apps.4ddbac784716b2f7da33.prod.konflux-ocp-ci.dev" "10.134.0.28:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.1.94:47090 10.133.1.94:443 10.132.0.21:40124 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T22:27:32.310Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 65 308 106 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "bdff2150-3be5-426e-96f1-74b4116dc7c8" "maas.apps.4ddbac784716b2f7da33.prod.konflux-ocp-ci.dev" "10.134.0.28:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.1.94:47100 10.133.1.94:443 10.132.0.21:40130 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T22:27:46.705991Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:27:57.428465Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:28:17.112599Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:28:24.421604Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:30:35.934526Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:30:35.934565Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:30:35.934590Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T22:30:35.934596Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T22:30:35.823Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 80 323 110 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "6d0377c0-092c-4206-b3b5-d3ed1694cd3a" "maas.apps.4ddbac784716b2f7da33.prod.konflux-ocp-ci.dev" "10.134.0.28:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.1.94:60500 10.133.1.94:443 10.132.0.21:41258 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T22:30:47.148Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 49 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "e6ec83ba-c165-4792-b537-006d3a760792" "maas.apps.4ddbac784716b2f7da33.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.1.94:443 10.132.0.21:58688 - v1-completions-path 2026-11-03T22:30:51.504290Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:32:18.984920Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:32:46.557566Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:32:51.258846Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:33:04.036066Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:33:21.001270Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:33:25.696685Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:33:46.145300Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:33:56.678729Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:34:02.693924Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T22:34:13.751Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 62 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "18a8d332-8c6f-47b7-9dce-192e95bb8a6e" "maas.apps.4ddbac784716b2f7da33.prod.konflux-ocp-ci.dev" "10.134.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.94:39114 10.133.1.94:443 10.132.0.21:34906 - v1-completions-path 2026-11-03T22:34:29.868587Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:34:36.025506Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:34:42.058070Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T22:34:48.959Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 57 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "4716dd52-190a-4c87-94e4-9c870cb34714" "maas.apps.4ddbac784716b2f7da33.prod.konflux-ocp-ci.dev" "10.134.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.94:37198 10.133.1.94:443 10.132.0.21:35516 - v1-completions-path [2026-11-03T22:34:49.839Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 56 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "1b10bab7-0451-4b7a-b364-d2211f0948ab" "maas.apps.4ddbac784716b2f7da33.prod.konflux-ocp-ci.dev" "10.134.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.94:37198 10.133.1.94:443 10.132.0.21:35520 - v1-completions-path [2026-11-03T22:34:56.736Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 65 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "3fef03c4-e579-4037-9077-4bf2eea6cfb8" "maas.apps.4ddbac784716b2f7da33.prod.konflux-ocp-ci.dev" "10.134.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.94:39114 10.133.1.94:443 10.132.0.21:37954 - v1-completions-path 2026-11-03T22:35:12.591129Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:35:23.297312Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:35:29.598334Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:35:35.429022Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:35:47.888558Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T22:35:56.575Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 61 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "8c4801a4-284f-4a8c-9bc6-2750144ff506" "maas.apps.4ddbac784716b2f7da33.prod.konflux-ocp-ci.dev" "10.134.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.94:37198 10.133.1.94:443 10.132.0.21:58136 - v1-completions-path 2026-11-03T22:36:11.272211Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:36:21.975499Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:36:26.793417Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:36:45.026415Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:36:51.192687Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:37:12.832115Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:37:34.783791Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:37:56.808001Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:38:16.598440Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:38:27.473473Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:38:38.403964Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:38:49.077246Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:38:52.021321Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-03T22:38:59.866865Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:39:10.041788Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:39:10.041820Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:39:10.041834Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:39:10.041839Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T22:39:10.559339Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T22:39:09.923Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 88 331 118 - "34.228.250.194,10.132.0.21" "python-requests/2.34.2" "e09913ed-fe87-4a94-bfd5-3ad549758a22" "maas.apps.4ddbac784716b2f7da33.prod.konflux-ocp-ci.dev" "10.132.0.166:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.1.94:51330 10.133.1.94:443 10.132.0.21:49698 - odh-ai-gateway-infra.maas-api-route.3