2026-11-02T02:55:39.086441Z info FLAG: --concurrency="0" 2026-11-02T02:55:39.086462Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-02T02:55:39.086466Z info FLAG: --help="false" 2026-11-02T02:55:39.086469Z info FLAG: --log_as_json="false" 2026-11-02T02:55:39.086471Z info FLAG: --log_caller="" 2026-11-02T02:55:39.086473Z info FLAG: --log_output_level="default:info" 2026-11-02T02:55:39.086475Z info FLAG: --log_stacktrace_level="default:none" 2026-11-02T02:55:39.086479Z info FLAG: --log_target="[stdout]" 2026-11-02T02:55:39.086482Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-02T02:55:39.086483Z info FLAG: --outlierLogPath="" 2026-11-02T02:55:39.086485Z info FLAG: --profiling="true" 2026-11-02T02:55:39.086487Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-02T02:55:39.086489Z info FLAG: --proxyLogLevel="warning" 2026-11-02T02:55:39.086491Z info FLAG: --serviceCluster="istio-proxy" 2026-11-02T02:55:39.086493Z info FLAG: --stsPort="0" 2026-11-02T02:55:39.086495Z info FLAG: --templateFile="" 2026-11-02T02:55:39.086497Z info FLAG: --tokenManagerPlugin="" 2026-11-02T02:55:39.086499Z info FLAG: --vklog="0" 2026-11-02T02:55:39.086502Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-02T02:55:39.086651Z info Proxy role ips=[10.134.0.166] type=router id=maas-default-gateway-openshift-default-769df7846c-vgqhz.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-02T02:55:39.086701Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-02T02:55:39.087910Z info cpu limit detected as 2, setting concurrency 2026-11-02T02:55:39.088160Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-02T02:55:39.088171Z info JWT policy is third-party-jwt 2026-11-02T02:55:39.088174Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-02T02:55:39.089586Z info Starting default Istio SDS Server 2026-11-02T02:55:39.089611Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-02T02:55:39.089650Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-02T02:55:39.090343Z info Opening status port 15020 2026-11-02T02:55:39.091146Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-02T02:55:39.091725Z info sds Starting SDS grpc server 2026-11-02T02:55:39.091739Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-02T02:55:39.092561Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-02T02:55:39.093488Z info Starting proxy agent 2026-11-02T02:55:39.093528Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-02T02:55:39.156849Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-02T02:55:39.237324Z info cache generated new workload certificate resourceName=default latency=145.489319ms ttl=23h59m59.762679856s 2026-11-02T02:55:39.237369Z info cache Root cert has changed, start rotating root cert 2026-11-02T02:55:39.237399Z info cache returned workload trust anchor from cache ttl=23h59m59.762601094s 2026-11-02T02:55:39.301201Z info ads ADS: new connection for node:1 2026-11-02T02:55:39.301274Z info cache returned workload certificate from cache ttl=23h59m59.698728511s 2026-11-02T02:55:39.301657Z info ads ADS: new connection for node:2 2026-11-02T02:55:39.301726Z info cache returned workload trust anchor from cache ttl=23h59m59.698275197s 2026-11-02T02:55:39.384852Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:55:42.868878Z error failed scraping envoy metrics: error scraping http://localhost:15090/stats/prometheus: Get "http://localhost:15090/stats/prometheus": dial tcp [::1]:15090: connect: connection refused 2026-11-02T02:55:47.315374Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:55:48.268100Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-02T02:55:48.596528Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-02T02:55:49.326888Z info Readiness succeeded in 10.243171765s 2026-11-02T02:55:49.327215Z info Envoy proxy is ready 2026-11-02T02:55:55.896540Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:56:07.433113Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:56:09.713557Z info ads ADS: new connection for node:3 2026-11-02T02:56:09.713749Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-02T02:56:09.713785Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-02T02:56:12.616041Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:56:58.455309Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:57:22.186666Z info ads ADS: "" 3 terminated 2026-11-02T02:57:22.474562Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:57:27.149155Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T02:57:43.551540Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:00:36.584027Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:01:04.784303Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:01:30.262368Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:01:45.812610Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:01:59.525010Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:02:12.190345Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:02:17.088150Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T03:02:21.928Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 90 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "97874d38-2614-4abd-a26b-a3686668f3ad" "maas.apps.82bac16f36aa9aea1c00.prod.konflux-ocp-ci.dev" "10.132.0.49:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.166:58634 10.134.0.166:443 10.132.0.10:52742 - v1-completions-path 2026-11-02T03:02:36.311010Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:03:23.643678Z info ads ADS: new connection for node:4 2026-11-02T03:03:23.648818Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-02T03:03:23.648870Z info cache read certificate from file resource=file-root:system 2026-11-02T03:03:26.822962Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:03:29.553368Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:03:41.470540Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"145","StreamId":"6447713598692364158"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=25 [2026-11-02T03:03:41.400Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 70 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "9b7b5dce-a86d-43c6-bbc2-847d4627460e" "maas.apps.82bac16f36aa9aea1c00.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.134.0.166:443 10.132.0.10:39024 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-11-02T03:03:46.218163Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:03:46.218197Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:03:46.218208Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T03:03:46.218212Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:03:46.218370Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-02T03:03:46.019Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1713 198 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "4c7451d7-5bed-4fd6-9564-edd33806f2f8" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.166:46150 10.134.0.166:443 10.132.0.10:39030 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T03:04:02.854669Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:04:02.854704Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:04:02.854713Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:04:02.854717Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T03:04:02.854859Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-02T03:04:02.727Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1717 127 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "42d7d7f9-08eb-4f3d-bb64-15ffdb683bd2" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.166:46150 10.134.0.166:443 10.132.0.10:55058 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T03:04:05.116557Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-11-02T03:04:05.184345Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:04:05.184376Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:04:05.184386Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:04:05.184391Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T03:04:05.184587Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-02T03:04:05.057Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1715 126 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "0a94fd55-fe71-46aa-adb1-ee94f000dfa1" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.166:46150 10.134.0.166:443 10.132.0.10:55068 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T03:04:16.948539Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:04:16.948576Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:04:16.948586Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T03:04:16.948590Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:04:16.948756Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-02T03:04:16.823Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1708 125 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "239dac7f-d887-4fc0-9c61-93690fd18110" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.166:46150 10.134.0.166:443 10.132.0.10:34152 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T03:04:30.947479Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:04:30.947516Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:04:30.947526Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:04:30.947530Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T03:04:30.947688Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-02T03:04:30.812Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1716 135 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "769a2388-9f8e-4751-bdb8-109e4ea59378" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.166:46150 10.134.0.166:443 10.132.0.10:40372 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T03:04:33.346985Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T03:04:33.347020Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T03:04:33.347031Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T03:04:33.347035Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T03:04:33.347251Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-02T03:04:33.148Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1715 199 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "8bc14258-ab8d-4935-8caf-9aa763c69738" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.166:54928 10.134.0.166:443 10.132.0.10:40382 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T03:04:35.668364Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:04:35.668399Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:04:35.668409Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:04:35.668413Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T03:04:35.668563Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-02T03:04:35.540Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1715 128 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "b948d703-52d1-45a1-aae8-3d1b1c1201b4" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.166:46150 10.134.0.166:443 10.132.0.10:40400 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T03:04:47.497700Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:04:47.497736Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:04:47.497746Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:04:47.497751Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T03:04:47.497953Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-02T03:04:47.369Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1713 128 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "bb27d472-bbc4-495e-9137-04d55cfe315b" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.166:46150 10.134.0.166:443 10.132.0.10:59860 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T03:04:49.819011Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T03:04:49.819039Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T03:04:49.819048Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T03:04:49.819052Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T03:04:49.819208Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-02T03:04:49.698Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1706 120 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "0a283548-0d2c-4328-a84c-796c004eca2a" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.166:54928 10.134.0.166:443 10.132.0.10:59870 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T03:04:52.143103Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:04:52.143133Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:04:52.143143Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T03:04:52.143147Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T03:04:52.143298Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-02T03:04:52.012Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1703 130 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "ab32472d-6b17-423a-a92a-8caa72bd9278" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.166:46150 10.134.0.166:443 10.132.0.10:59898 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T03:05:07.238635Z info ads ADS: "" 4 terminated 2026-11-02T03:05:07.281755Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:05:09.946836Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:05:20.515185Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:05:38.534087Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:05:43.119912Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:05:46.120053Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-11-02T03:05:46.058Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 288 81 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "1cc9de10-3cf4-4859-a78c-66f5a9bdcd77" "maas.apps.82bac16f36aa9aea1c00.prod.konflux-ocp-ci.dev" "10.132.0.49:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.166:48186 10.134.0.166:443 10.132.0.10:38954 - v1-completions-path [2026-11-02T03:05:46.924Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 64 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "2bb5ca80-731c-4df5-a3f5-76106d8bd5c3" "maas.apps.82bac16f36aa9aea1c00.prod.konflux-ocp-ci.dev" "10.132.0.49:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.166:58634 10.134.0.166:443 10.132.0.10:60214 - v1-completions-path [2026-11-02T03:05:47.175Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 300 61 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "90107903-d4cd-47d3-9d85-c650bb376722" "maas.apps.82bac16f36aa9aea1c00.prod.konflux-ocp-ci.dev" "10.132.0.49:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.166:48186 10.134.0.166:443 10.132.0.10:60222 - v1-completions-path [2026-11-02T03:05:48.709Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 61 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "9215e71a-1bb1-41ad-b467-a7d155455876" "maas.apps.82bac16f36aa9aea1c00.prod.konflux-ocp-ci.dev" "10.132.0.49:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.166:48186 10.134.0.166:443 10.132.0.10:60232 - v1-completions-path 2026-11-02T03:06:03.253195Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:06:21.785400Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T03:06:30.220Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 67 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "02f5266f-bd94-4172-a3e2-426a52651511" "maas.apps.82bac16f36aa9aea1c00.prod.konflux-ocp-ci.dev" "10.132.0.49:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.166:58634 10.134.0.166:443 10.132.0.10:41740 - v1-completions-path 2026-11-02T03:06:53.666851Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:07:04.618002Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:07:10.743791Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T03:07:22.365Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 64 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "04ab990c-923d-4bce-b225-7bb0e4722c0c" "maas.apps.82bac16f36aa9aea1c00.prod.konflux-ocp-ci.dev" "10.132.0.49:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.166:48186 10.134.0.166:443 10.132.0.10:36982 - v1-completions-path [2026-11-02T03:07:24.120Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 58 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "7f192f82-4443-4678-9be8-5ca79230fea5" "maas.apps.82bac16f36aa9aea1c00.prod.konflux-ocp-ci.dev" "10.132.0.49:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.166:48186 10.134.0.166:443 10.132.0.10:36990 - v1-completions-path 2026-11-02T03:07:38.866523Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:07:43.414520Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:07:56.719385Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:08:05.492863Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:08:13.375418Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:08:23.299399Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:08:32.760708Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:08:51.992073Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:09:23.006468Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:09:45.716718Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:09:56.497614Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:14:15.751105Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:14:35.528594Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:14:40.169682Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:14:53.187967Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:15:09.502545Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:15:14.793526Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:15:18.086076Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T03:15:18.086113Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T03:15:18.086128Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T03:15:18.086133Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T03:15:17.964Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 72 315 121 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "84f8bb7c-2498-4a13-83ab-b9387e98ce4c" "maas.apps.82bac16f36aa9aea1c00.prod.konflux-ocp-ci.dev" "10.133.0.228:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.166:43582 10.134.0.166:443 10.132.0.10:48146 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T03:15:18.558Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 61 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "5285272c-c7ab-487f-9c89-dfeab8af71fa" "maas.apps.82bac16f36aa9aea1c00.prod.konflux-ocp-ci.dev" "10.132.0.49:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.166:48186 10.134.0.166:443 10.132.0.10:48156 - v1-completions-path 2026-11-02T03:15:34.861601Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:15:45.525220Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:15:51.809590Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:16:21.869601Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:16:28.042545Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:16:34.530468Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:17:06.235191Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:17:17.209711Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:17:23.788391Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:17:30.036386Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:17:41.647784Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T03:17:49.589Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 60 - "44.212.242.249,10.132.0.10" "python-requests/2.34.2" "599edaf9-755c-453a-b999-78db5cfadc9d" "maas.apps.82bac16f36aa9aea1c00.prod.konflux-ocp-ci.dev" "10.132.0.49:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.166:58634 10.134.0.166:443 10.132.0.10:41114 - v1-completions-path 2026-11-02T03:18:04.949094Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:18:15.593038Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:18:20.583412Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:18:38.300277Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:18:44.707219Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:19:08.015140Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:19:30.325269Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:19:52.011817Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:20:12.158198Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:20:23.162898Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:20:33.833590Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:20:44.670673Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:20:55.470446Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T03:21:06.031434Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8