2026-10-31T04:09:28.946364Z info FLAG: --concurrency="0" 2026-10-31T04:09:28.946385Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-31T04:09:28.946389Z info FLAG: --help="false" 2026-10-31T04:09:28.946392Z info FLAG: --log_as_json="false" 2026-10-31T04:09:28.946394Z info FLAG: --log_caller="" 2026-10-31T04:09:28.946396Z info FLAG: --log_output_level="default:info" 2026-10-31T04:09:28.946398Z info FLAG: --log_stacktrace_level="default:none" 2026-10-31T04:09:28.946403Z info FLAG: --log_target="[stdout]" 2026-10-31T04:09:28.946405Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-31T04:09:28.946407Z info FLAG: --outlierLogPath="" 2026-10-31T04:09:28.946409Z info FLAG: --profiling="true" 2026-10-31T04:09:28.946411Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-31T04:09:28.946413Z info FLAG: --proxyLogLevel="warning" 2026-10-31T04:09:28.946415Z info FLAG: --serviceCluster="istio-proxy" 2026-10-31T04:09:28.946417Z info FLAG: --stsPort="0" 2026-10-31T04:09:28.946419Z info FLAG: --templateFile="" 2026-10-31T04:09:28.946421Z info FLAG: --tokenManagerPlugin="" 2026-10-31T04:09:28.946423Z info FLAG: --vklog="0" 2026-10-31T04:09:28.946426Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-31T04:09:28.946565Z info Proxy role ips=[10.133.0.158] type=router id=maas-default-gateway-openshift-default-6886b96f9-lxq9w.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-31T04:09:28.946604Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-31T04:09:28.947814Z info cpu limit detected as 2, setting concurrency 2026-10-31T04:09:28.948217Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-31T04:09:28.948230Z info JWT policy is third-party-jwt 2026-10-31T04:09:28.948233Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-31T04:09:28.950360Z info Starting default Istio SDS Server 2026-10-31T04:09:28.950385Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-31T04:09:28.950405Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-31T04:09:28.950562Z info Opening status port 15020 2026-10-31T04:09:28.951177Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-31T04:09:28.952456Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-31T04:09:28.953531Z info Starting proxy agent 2026-10-31T04:09:28.953572Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-31T04:09:28.962306Z info sds Starting SDS grpc server 2026-10-31T04:09:28.962325Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-31T04:09:29.018404Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-31T04:09:29.089180Z info cache generated new workload certificate resourceName=default latency=137.852015ms ttl=23h59m59.910824615s 2026-10-31T04:09:29.089223Z info cache Root cert has changed, start rotating root cert 2026-10-31T04:09:29.089257Z info cache returned workload trust anchor from cache ttl=23h59m59.910743382s 2026-10-31T04:09:29.262816Z info ads ADS: new connection for node:1 2026-10-31T04:09:29.269171Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-10-31T04:09:29.269220Z info cache read certificate from file resource=file-root:system 2026-10-31T04:09:29.307952Z info ads ADS: new connection for node:2 2026-10-31T04:09:29.308024Z info cache returned workload certificate from cache ttl=23h59m59.691978612s 2026-10-31T04:09:29.309485Z info ads ADS: new connection for node:3 2026-10-31T04:09:29.309564Z info cache returned workload trust anchor from cache ttl=23h59m59.690438144s 2026-10-31T04:09:29.407988Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:09:37.486072Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:09:39.555678Z info Readiness succeeded in 10.611598246s 2026-10-31T04:09:39.555702Z info ads ADS: "" 1 terminated with stream closed 2026-10-31T04:09:39.556031Z info Envoy proxy is ready 2026-10-31T04:09:39.563898Z info Readiness succeeded in 10.619824945s 2026-10-31T04:09:39.635004Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:09:39.679523Z info ads ADS: new connection for node:4 2026-10-31T04:09:39.684623Z info cache read certificate from file resource=file-root:system 2026-10-31T04:09:40.820405Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:09:40.853335Z info ads ADS: "" 4 terminated with stream closed 2026-10-31T04:09:41.784344Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:09:48.806440Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:10:34.633950Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:17:58.054382Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:19:31.634842Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:19:59.712665Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:21:39.667386Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:30:34.018997Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:30:34.315563Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T04:30:34.315612Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T04:30:34.315633Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T04:30:34.315640Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-31T04:30:34.180Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 313 134 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "fed41881-1688-473e-b621-008d87aa7158" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.22:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.158:60712 10.133.0.158:443 10.132.0.15:35752 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T04:31:01.471520Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:31:40.083486Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-31T04:31:45.214Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 140 405 78 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "01330244-0ed8-4de1-ad9c-69c7e8f1037c" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.32:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.133.0.158:38582 10.133.0.158:443 10.132.0.15:59014 - v1-chat-completions-path 2026-10-31T04:31:58.870578Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:32:02.252055Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:32:11.789278Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:32:17.488745Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-10-31T04:32:17.412Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 76 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "e8b35805-e637-42b7-8168-6875dc3cd844" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.158:54228 10.133.0.158:443 10.132.0.15:51756 - v1-catch-all-path 2026-10-31T04:32:25.476561Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:32:34.662537Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:32:51.541384Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:33:03.322825Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:33:07.923014Z info ads ADS: new connection for node:5 2026-10-31T04:33:07.923219Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-31T04:33:07.923247Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-31T04:33:14.891911Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:33:54.221815Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-31T04:33:57.610Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 341 100 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "e44a38d7-ab3e-4a15-93e5-9acae4eebbbb" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.202:8000" outbound|54321||e2e-prio-pool-7010fecc-inference-pool-ip-529cbf0f.llm.svc.cluster.local 10.133.0.158:36622 10.133.0.158:443 10.132.0.15:42008 - v1-model-routing [2026-10-31T04:33:58.538Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 349 59 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "b18fd7dd-771d-431f-8b81-8b6076aa1e36" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.202:8000" outbound|54321||e2e-prio-pool-7010fecc-inference-pool-ip-529cbf0f.llm.svc.cluster.local 10.133.0.158:36622 10.133.0.158:443 10.132.0.15:42030 - v1-model-routing 2026-10-31T04:34:00.591432Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-10-31T04:34:00.533Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 328 58 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "2fea3453-39ed-4b29-9b4a-e24ca6d1a2e0" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.202:8000" outbound|54321||e2e-prio-pool-7010fecc-inference-pool-ip-529cbf0f.llm.svc.cluster.local 10.133.0.158:36622 10.133.0.158:443 10.132.0.15:42038 - v1-model-routing 2026-10-31T04:34:18.268210Z info ads ADS: "" 5 terminated 2026-10-31T04:34:18.577170Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:34:23.456481Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-31T04:34:23.179Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 289 86 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "b38e19c3-392e-4830-8b32-55b2366a5893" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.158:58090 10.133.0.158:443 10.132.0.15:43876 - v1-completions-path 2026-10-31T04:34:40.476513Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:36:53.854698Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T04:36:53.854736Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T04:36:53.854756Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T04:36:53.854762Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-31T04:36:53.725Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 33 323 129 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "9fedf1cd-76df-4615-b456-32c7e90445ab" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.22:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.158:54152 10.133.0.158:443 10.132.0.15:35880 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T04:37:11.108046Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:37:33.830670Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:38:01.249357Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:38:12.821091Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:38:19.552413Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:38:32.595482Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:38:37.402544Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:38:40.329049Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-10-31T04:38:40.329126Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T04:38:40.329149Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T04:38:40.329165Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T04:38:40.329171Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T04:38:40.563704Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T04:38:40.563737Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T04:38:40.563753Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T04:38:40.563758Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-31T04:38:39.916Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 335 412 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "6cdfa8a9-5480-42a0-8544-f6d8950b53b5" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.22:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.158:38994 10.133.0.158:443 10.132.0.15:38720 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T04:38:40.514Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 524 49 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "b12fce85-632d-4375-a48b-c0ed18eb5e0f" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.22:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.158:38994 10.133.0.158:443 10.132.0.15:38722 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T04:38:55.340818Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:39:43.487990Z info ads ADS: new connection for node:6 2026-10-31T04:39:43.491345Z info cache read certificate from file resource=file-root:system 2026-10-31T04:39:47.387094Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:39:48.423617Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:39:56.217934Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"523","StreamId":"6440179658268481706"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=25 [2026-10-31T04:39:56.154Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 63 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "bac71e81-70a5-4e7e-8121-36627ca5ee8c" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.133.0.158:443 10.132.0.15:51274 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-10-31T04:39:58.485035Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"526","StreamId":"6169167758794182394"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=24 [2026-10-31T04:39:58.425Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 59 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "062d50ca-715b-4d5e-85a5-da2fbbfc112f" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.133.0.158:443 10.132.0.15:51282 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-10-31T04:40:32.074411Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T04:40:32.074446Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T04:40:32.074457Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T04:40:32.074461Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T04:40:32.074615Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-31T04:40:31.844Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1710 229 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "ca533f7c-2a0b-4f87-90a5-243edebfa4b0" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.158:33038 10.133.0.158:443 10.132.0.15:55716 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-31T04:40:51.022672Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T04:40:51.022702Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T04:40:51.022712Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T04:40:51.022716Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T04:40:51.022883Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-31T04:40:50.858Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1712 164 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "d9d8775e-42c3-4cc5-a21c-3fc16d11556f" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.158:33038 10.133.0.158:443 10.132.0.15:50804 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-31T04:40:58.006951Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T04:40:58.006981Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T04:40:58.006990Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T04:40:58.006995Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T04:40:58.007148Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-31T04:40:57.874Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1714 132 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "fc5a8489-d9a3-4228-a321-236d3f837cbe" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.158:33038 10.133.0.158:443 10.132.0.15:55864 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-31T04:41:00.345319Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T04:41:00.345350Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T04:41:00.345360Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T04:41:00.345364Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T04:41:00.345509Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-31T04:41:00.215Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1712 130 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "e9577f57-14b0-4733-aafc-f2e7854f7886" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.158:33038 10.133.0.158:443 10.132.0.15:55874 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-31T04:41:12.018078Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-10-31T04:41:21.734596Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T04:41:21.734631Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T04:41:21.734644Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T04:41:21.734648Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T04:41:21.734845Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-10-31T04:41:21.538Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1711 196 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "f28e58bd-03c7-40b3-aa9b-8238bd2def13" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.158:37980 10.133.0.158:443 10.132.0.15:50056 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-31T04:41:43.904827Z info ads ADS: "" 6 terminated with stream closed 2026-10-31T04:41:43.958201Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:41:46.647537Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:41:57.059165Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:41:59.506833Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T04:41:59.506868Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T04:41:59.506883Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T04:41:59.506888Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-31T04:41:57.691Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 87 329 1815 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "8ce0db16-7fb7-47aa-b4e1-e14d0c7a2e6e" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.22:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.158:60146 10.133.0.158:443 10.132.0.15:40754 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T04:42:16.356677Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:42:20.927238Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-31T04:42:24.778Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 71 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "da251762-3166-46a5-830d-385141ecda96" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.158:39466 10.133.0.158:443 10.132.0.15:34736 - v1-completions-path [2026-10-31T04:42:25.583Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 60 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "f78bf679-47e5-467e-977d-5d6357ea9eae" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.158:47360 10.133.0.158:443 10.132.0.15:34746 - v1-completions-path [2026-10-31T04:42:39.911Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 288 67 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "fe67be6f-3897-4628-b1d0-f871d1e5aa6e" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.158:47734 10.133.0.158:443 10.132.0.15:56282 - v1-completions-path 2026-10-31T04:42:45.017557Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:42:59.760817Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-31T04:43:10.831Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 65 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "27a6a4d2-7c35-4150-850b-7260d7d85096" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.158:39466 10.133.0.158:443 10.132.0.15:43398 - v1-completions-path [2026-10-31T04:43:11.104Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 59 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "7d9ed6ca-f978-4475-b9f7-a1084d94ec7b" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.158:39466 10.133.0.158:443 10.132.0.15:43400 - v1-completions-path 2026-10-31T04:43:31.930468Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:43:42.588145Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:43:48.958027Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-31T04:44:01.191Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 57 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "f1a5817d-6f78-4e19-98cb-45bdaabf3090" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.158:47360 10.133.0.158:443 10.132.0.15:54634 - v1-completions-path [2026-10-31T04:44:02.725Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 60 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "c1465748-ea00-4929-9696-97cbfe39964b" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.158:39466 10.133.0.158:443 10.132.0.15:54644 - v1-completions-path 2026-10-31T04:44:16.974508Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:44:21.326729Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:44:32.697351Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:44:41.161760Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:44:51.321461Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:44:59.768295Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:45:08.874778Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:45:27.051485Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:45:32.983098Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:45:53.294976Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:45:58.354894Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:46:10.647328Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:46:33.263997Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:46:44.059147Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:47:03.014223Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:47:10.576021Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:49:36.280521Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:51:05.300363Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:51:22.616816Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:51:27.727493Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:51:30.636281Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-31T04:51:30.636309Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-31T04:51:30.636312Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-31T04:51:30.636316Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-10-31T04:51:30.368Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 123 267 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "601ea8e4-292d-498d-83f0-ec6879fd3e6c" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.158:443 10.132.0.15:51406 - v1-completions-path 2026-10-31T04:51:41.225598Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:51:56.724620Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:52:00.381310Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:52:29.641601Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:52:40.381342Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:52:46.559471Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-31T04:53:01.810Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 62 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "4373ec0f-b264-46a1-85e8-02581e87d658" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.158:47360 10.133.0.158:443 10.132.0.15:34322 - v1-completions-path 2026-10-31T04:53:16.894414Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:53:23.302040Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:53:29.776308Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-31T04:53:35.193Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 63 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "71a34625-4831-4739-bb23-bd590fe9a8b5" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.158:47360 10.133.0.158:443 10.132.0.15:56340 - v1-completions-path [2026-10-31T04:53:37.399Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 62 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "249a2c21-11d8-456d-91ec-79f3caa5f71f" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.158:47360 10.133.0.158:443 10.132.0.15:56350 - v1-completions-path [2026-10-31T04:53:38.848Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 65 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "44947b52-4558-4f2d-be71-abb6de3ee65e" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.158:39466 10.133.0.158:443 10.132.0.15:56358 - v1-completions-path [2026-10-31T04:53:39.219Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 65 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "2986c91b-fb12-4ee9-b492-2a821b19c158" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.158:39466 10.133.0.158:443 10.132.0.15:56366 - v1-completions-path [2026-10-31T04:53:39.961Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 67 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "60591d50-c6dc-49cc-9556-520a4211dee6" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.158:39466 10.133.0.158:443 10.132.0.15:56382 - v1-completions-path [2026-10-31T04:53:41.036Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 68 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "4e0bc4f3-b82e-4034-aa54-35c264e4e310" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.158:47360 10.133.0.158:443 10.132.0.15:56392 - v1-completions-path [2026-10-31T04:53:41.779Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 66 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "89e81213-4570-4e3d-b443-44127750f616" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.158:39466 10.133.0.158:443 10.132.0.15:56408 - v1-completions-path 2026-10-31T04:53:58.669219Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:54:09.343172Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:54:16.080531Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:54:22.296829Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:54:32.067076Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-31T04:54:40.883Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 61 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "55153bca-7bd7-4bd4-94e5-ae1b902360fb" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.158:47360 10.133.0.158:443 10.132.0.15:36498 - v1-completions-path [2026-10-31T04:54:41.608Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 62 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "3c47d6ea-ee2b-479b-8083-7c44e0912d86" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.158:39466 10.133.0.158:443 10.132.0.15:36502 - v1-completions-path 2026-10-31T04:54:57.154710Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:55:07.931505Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:55:12.876587Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:55:30.919274Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:55:37.434251Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:55:58.482759Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:56:18.400129Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:56:29.358160Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:56:40.227040Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:56:51.286050Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:57:01.952438Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:57:12.892244Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:57:23.446328Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:57:34.260385Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:57:45.021457Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T04:57:55.756225Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-31T04:57:57.398Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 388 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "ad52a49a-758f-4018-857d-96fac97a0b22" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.158:47734 10.133.0.158:443 10.132.0.15:38844 - v1-completions-path [2026-10-31T04:58:06.952Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 283 72 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "634b1f11-ee7f-42f8-b27a-fafb59391f09" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.158:58090 10.133.0.158:443 10.132.0.15:48448 - v1-completions-path 2026-10-31T04:58:08.077069Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T04:58:08.077093Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T04:58:08.077096Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T04:58:08.077100Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-10-31T04:58:08.051Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 401 - - "-" 66 88 25 - "52.71.36.72,10.132.0.15" "python-requests/2.34.2" "40f6a62d-ab52-4d09-b14a-5c8ad1b3d3cd" "maas.apps.6b93ef1713d6ca59dc0c.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.158:443 10.132.0.15:48470 - v1-completions-path 2026-10-31T04:58:17.247363Z info cache event for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt : REMOVE, pushing to proxy