2026-11-02T09:44:22.484188Z info FLAG: --concurrency="0" 2026-11-02T09:44:22.484210Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-02T09:44:22.484231Z info FLAG: --help="false" 2026-11-02T09:44:22.484234Z info FLAG: --log_as_json="false" 2026-11-02T09:44:22.484236Z info FLAG: --log_caller="" 2026-11-02T09:44:22.484238Z info FLAG: --log_output_level="default:info" 2026-11-02T09:44:22.484240Z info FLAG: --log_stacktrace_level="default:none" 2026-11-02T09:44:22.484246Z info FLAG: --log_target="[stdout]" 2026-11-02T09:44:22.484248Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-02T09:44:22.484250Z info FLAG: --outlierLogPath="" 2026-11-02T09:44:22.484252Z info FLAG: --profiling="true" 2026-11-02T09:44:22.484254Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-02T09:44:22.484256Z info FLAG: --proxyLogLevel="warning" 2026-11-02T09:44:22.484258Z info FLAG: --serviceCluster="istio-proxy" 2026-11-02T09:44:22.484260Z info FLAG: --stsPort="0" 2026-11-02T09:44:22.484262Z info FLAG: --templateFile="" 2026-11-02T09:44:22.484264Z info FLAG: --tokenManagerPlugin="" 2026-11-02T09:44:22.484275Z info FLAG: --vklog="0" 2026-11-02T09:44:22.484279Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-02T09:44:22.484444Z info Proxy role ips=[10.134.0.58] type=router id=maas-default-gateway-openshift-default-75fd9f5d47-qwf92.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-02T09:44:22.484490Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-02T09:44:22.486853Z info cpu limit detected as 2, setting concurrency 2026-11-02T09:44:22.487367Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-02T09:44:22.487388Z info JWT policy is third-party-jwt 2026-11-02T09:44:22.487394Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-02T09:44:22.489738Z info Starting default Istio SDS Server 2026-11-02T09:44:22.489763Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-02T09:44:22.489787Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-02T09:44:22.489722Z info Opening status port 15020 2026-11-02T09:44:22.490531Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-02T09:44:22.492419Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-02T09:44:22.493493Z info Starting proxy agent 2026-11-02T09:44:22.493590Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-02T09:44:22.500590Z info sds Starting SDS grpc server 2026-11-02T09:44:22.500632Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-02T09:44:22.586829Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-02T09:44:22.633964Z info cache generated new workload certificate resourceName=default latency=143.244885ms ttl=23h59m59.366039288s 2026-11-02T09:44:22.633999Z info cache Root cert has changed, start rotating root cert 2026-11-02T09:44:22.634033Z info cache returned workload trust anchor from cache ttl=23h59m59.365967826s 2026-11-02T09:44:22.714852Z info ads ADS: new connection for node:1 2026-11-02T09:44:22.714948Z info cache returned workload certificate from cache ttl=23h59m59.285055333s 2026-11-02T09:44:22.715676Z info ads ADS: new connection for node:2 2026-11-02T09:44:22.715762Z info cache returned workload trust anchor from cache ttl=23h59m59.284240982s 2026-11-02T09:44:22.815871Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T09:44:34.693388Z warn xdsproxy sendDownstreamDelta took 10.638367964s 2026-11-02T09:44:34.916640Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T09:44:35.714070Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-02T09:44:36.159520Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-02T09:44:37.164400Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-02T09:44:39.540418Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T09:44:42.415384Z warn Envoy proxy is NOT ready: failed to get readiness stats: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(server\\.state|listener_manager\\.workers_started)": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-02T09:44:42.484735Z warn Envoy proxy is NOT ready: failed to get readiness stats: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(server\\.state|listener_manager\\.workers_started)": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-02T09:44:42.682074Z info Readiness succeeded in 20.200429174s 2026-11-02T09:44:42.682619Z info Envoy proxy is ready 2026-11-02T09:46:17.545513Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-02T09:46:17.523Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 91 21 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "5b9f4240-ab74-484e-af36-a11f6fb204d7" "maas.apps.e930e379d4db695acbc5.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.58:36624 10.134.0.58:443 10.132.0.19:44334 - odh-ai-gateway-infra.maas-api-route.4 2026-11-02T09:48:05.384717Z info ads ADS: new connection for node:3 2026-11-02T09:48:05.390179Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-02T09:48:05.390249Z info cache read certificate from file resource=file-root:system 2026-11-02T09:54:28.049026Z info ads ADS: "" 3 terminated 2026-11-02T09:54:43.203573Z info ads ADS: new connection for node:4 2026-11-02T09:54:43.208983Z info cache read certificate from file resource=file-root:system 2026-11-02T09:55:30.735284Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T09:55:52.519358Z info ads ADS: "" 4 terminated with stream closed 2026-11-02T09:55:53.127952Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T09:55:58.965714Z info ads ADS: new connection for node:5 2026-11-02T09:55:58.983804Z info cache read certificate from file resource=file-root:system 2026-11-02T09:56:07.014465Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T09:56:35.234667Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T09:56:47.653949Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T09:56:58.120575Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T09:57:03.344805Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T09:57:03.345093Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-02T09:57:03.123Z] "POST /llm/e2e-rt-sw-b5d95f/anything/e2e-rt-sw-b5d95f-client HTTP/1.1" 503 UH no_healthy_upstream - "-" 124 19 221 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "d2b7cecd-7de2-4fbb-900f-4e574cd1862b" "maas.apps.e930e379d4db695acbc5.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rt-sw-b5d95f.llm.svc.cluster.local - 10.134.0.58:443 10.132.0.19:47468 - llm.external-model-e2e-rt-sw-b5d95f-5cff12787ff3.2 2026-11-02T09:57:09.484230Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T09:57:09.484570Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-02T09:57:09.257Z] "POST /anything/e2e-rt-sw-b5d95f-client HTTP/1.1" 200 - via_upstream - "-" 124 1770 227 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "e31c78c4-1a84-4268-8584-2a99837060bc" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.58:48144 10.134.0.58:443 10.132.0.19:36932 - llm.external-model-e2e-rt-sw-b5d95f-5cff12787ff3.0 2026-11-02T09:57:21.284092Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T09:57:25.315939Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T09:57:25.316195Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-02T09:57:25.178Z] "POST /anything/e2e-rt-sw-b5d95f-client HTTP/1.1" 200 - via_upstream - "-" 124 1770 137 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "2821f0d4-e625-4bb0-8b30-bc23af7ee5fe" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.58:48144 10.134.0.58:443 10.132.0.19:42874 - llm.external-model-e2e-rt-sw-b5d95f-5cff12787ff3.0 2026-11-02T09:57:35.553635Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T09:57:42.419301Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T09:57:49.671920Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T09:57:56.131906Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T09:57:56.132189Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-02T09:57:55.948Z] "POST /anything/e2e-rt-eq-6ae45e-client HTTP/1.1" 200 - via_upstream - "-" 124 1769 183 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "93158cee-c249-4744-8850-6c37619440ca" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.58:46084 10.134.0.58:443 10.132.0.19:46790 - llm.external-model-e2e-rt-eq-6ae45e-02f6e2c83742.0 2026-11-02T09:58:13.003876Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T09:58:21.339428Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T09:58:23.554606Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T09:58:30.035189Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T09:59:22.893506Z info ads ADS: "" 5 terminated with stream closed 2026-11-02T09:59:38.065622Z info ads ADS: new connection for node:6 2026-11-02T09:59:38.072997Z info cache read certificate from file resource=file-root:system 2026-11-02T09:59:38.775269Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T09:59:42.288802Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-02T09:59:59.276Z] "POST /llm/e2e-dft-cce475/anything/e2e-dft-cce475-a69a66 HTTP/1.1" 401 - - "-" 109 88 68 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "3f884567-31f1-4335-a38b-056100b098dd" "maas.apps.e930e379d4db695acbc5.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-cce475.llm.svc.cluster.local - 10.134.0.58:443 10.132.0.19:50306 - llm.external-model-e2e-dft-cce475-9623dcb9d9cd.2 2026-11-02T10:00:11.061165Z info ads ADS: "" 6 terminated with stream closed 2026-11-02T10:00:11.117071Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:00:13.239657Z info ads ADS: new connection for node:7 2026-11-02T10:00:13.244442Z info cache read certificate from file resource=file-root:system 2026-11-02T10:00:13.545621Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:00:13.888316Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:00:14.226747Z info ads ADS: "" 7 terminated 2026-11-02T10:00:14.536642Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:00:15.420826Z info ads ADS: new connection for node:8 2026-11-02T10:00:15.426478Z info cache read certificate from file resource=file-root:system 2026-11-02T10:00:15.738717Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:00:16.895405Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:00:16.925052Z info ads ADS: "" 8 terminated 2026-11-02T10:00:18.693500Z info ads ADS: new connection for node:9 2026-11-02T10:00:18.710628Z info cache read certificate from file resource=file-root:system 2026-11-02T10:00:19.237981Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:00:20.071401Z info ads ADS: "" 9 terminated 2026-11-02T10:00:20.117891Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:00:24.128772Z info ads ADS: new connection for node:10 2026-11-02T10:00:24.140510Z info cache read certificate from file resource=file-root:system 2026-11-02T10:01:29.947689Z info ads ADS: "" 10 terminated 2026-11-02T10:01:39.671443Z info ads ADS: new connection for node:11 2026-11-02T10:01:39.677877Z info cache read certificate from file resource=file-root:system 2026-11-02T10:02:55.115188Z info ads ADS: new connection for node:12 2026-11-02T10:02:55.115548Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-02T10:02:55.115614Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-02T10:03:40.508055Z info ads ADS: "" 12 terminated 2026-11-02T10:04:28.177620Z info ads ADS: "" 11 terminated 2026-11-02T10:05:54.099888Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:06:15.183942Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:06:47.650601Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-02T10:06:51.636Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 108118 108394 221 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "b77a2881-bb6d-425b-8ff8-9be16445d153" "maas.apps.e930e379d4db695acbc5.prod.konflux-ocp-ci.dev" "10.134.0.46:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.134.0.58:44512 10.134.0.58:443 10.132.0.19:37930 - v1-chat-completions-path 2026-11-02T10:07:04.446251Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:07:06.206545Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:07:10.364687Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-11-02T10:07:10.283Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 91 3616 80 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "5de62c76-cca3-4a0a-8e93-7dcca0cf1cc1" "maas.apps.e930e379d4db695acbc5.prod.konflux-ocp-ci.dev" "10.133.0.36:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.58:43258 10.134.0.58:443 10.132.0.19:48334 - v1-catch-all-model-routing 2026-11-02T10:07:12.738773Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:07:16.923246Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-11-02T10:07:16.861Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 62 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "708fdc95-0f1a-4118-a580-4afe5a1dc4b7" "maas.apps.e930e379d4db695acbc5.prod.konflux-ocp-ci.dev" "10.133.0.36:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.58:43258 10.134.0.58:443 10.132.0.19:58264 - v1-catch-all-path 2026-11-02T10:07:29.172945Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:07:31.125600Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:07:46.486062Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:07:57.456991Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:07:59.976939Z info ads ADS: new connection for node:13 2026-11-02T10:07:59.977239Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-02T10:08:02.551769Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:08:41.655929Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:09:01.665778Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:09:01.788258Z info ads ADS: "" 13 terminated with stream closed 2026-11-02T10:09:03.619206Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:09:06.267688Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:09:10.176261Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:10:50.538991Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:11:07.390490Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:11:23.423566Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:11:35.208568Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:11:39.280271Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:11:50.656726Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-02T10:11:51.018160Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:11:53.747349Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:11:57.265743Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-02T10:11:57.117Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 336 148 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "b2ced446-7b0a-4bf4-b9bd-3f5bcfb12b6e" "maas.apps.e930e379d4db695acbc5.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.58:42708 10.134.0.58:443 10.132.0.19:33908 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T10:11:57.576Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 77 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "2bb589dc-e042-4c6e-8300-381914a56c22" "maas.apps.e930e379d4db695acbc5.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.58:54736 10.134.0.58:443 10.132.0.19:54952 - v1-completions-path 2026-11-02T10:11:58.215197Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-02T10:11:58.156Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 524 59 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "4e379cb5-26cd-4758-817f-fe7ac86560a4" "maas.apps.e930e379d4db695acbc5.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.58:42708 10.134.0.58:443 10.132.0.19:54960 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T10:12:09.736814Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:15:58.171838Z info ads ADS: new connection for node:14 2026-11-02T10:15:58.177201Z info cache read certificate from file resource=file-root:system 2026-11-02T10:15:58.814578Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:16:00.411180Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:16:08.690332Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-02T10:16:08.744011Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"531","StreamId":"17730314162808444354"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=25 [2026-11-02T10:16:08.679Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 64 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "8386d011-a7a9-498c-ba88-927f43732e00" "maas.apps.e930e379d4db695acbc5.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.134.0.58:443 10.132.0.19:60386 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-11-02T10:16:15.418499Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T10:16:15.418765Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-02T10:16:15.284Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1719 134 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "496d9e35-3160-4d1d-8c40-5b213f53a198" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.58:45348 10.134.0.58:443 10.132.0.19:60418 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T10:16:17.608547Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T10:16:17.608821Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-02T10:16:17.476Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1718 132 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "1f2902e1-8921-4458-9116-38aab0daa64e" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.58:45364 10.134.0.58:443 10.132.0.19:37242 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T10:16:46.008964Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T10:16:46.009301Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-02T10:16:45.899Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1722 109 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "5cd77994-5441-4c51-ab2b-2597769d8ef0" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.58:45348 10.134.0.58:443 10.132.0.19:56238 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T10:16:56.889645Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T10:16:56.889936Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-02T10:16:56.782Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1719 107 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "beec59d0-eec3-4211-a682-34ff0844a455" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.58:45364 10.134.0.58:443 10.132.0.19:37352 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T10:17:01.214528Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T10:17:01.214840Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-02T10:17:01.113Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1721 101 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "200f988a-310d-4562-97b7-61b8bb2bd4c0" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.58:45364 10.134.0.58:443 10.132.0.19:37376 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T10:17:18.565530Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T10:17:18.565835Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-02T10:17:18.461Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1716 104 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "84975606-e631-4991-960d-df09096e4e18" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.58:45364 10.134.0.58:443 10.132.0.19:59952 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T10:17:31.807904Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:17:31.928290Z info ads ADS: "" 14 terminated with stream closed 2026-11-02T10:17:33.620484Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:17:36.819918Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:17:37.517361Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T10:17:37.419Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 87 329 97 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "4f519abd-6ea2-4bbb-9d8d-0c09b4e8d5db" "maas.apps.e930e379d4db695acbc5.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.58:56678 10.134.0.58:443 10.132.0.19:37884 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T10:17:49.134200Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:17:50.851032Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:18:06.767756Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:18:08.446457Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:18:12.262076Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:18:19.517461Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-02T10:18:26.794Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 62 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "1cc67304-47ac-45f9-abb5-b7792dcc5b19" "maas.apps.e930e379d4db695acbc5.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.58:443 10.132.0.19:43412 - v1-completions-path 2026-11-02T10:18:44.348812Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:18:54.537610Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:18:56.807713Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-02T10:19:02.151Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 65 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "80babbbc-df63-463a-9f93-5cb49a7bc4b8" "maas.apps.e930e379d4db695acbc5.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.58:54736 10.134.0.58:443 10.132.0.19:49442 - v1-completions-path 2026-11-02T10:19:15.386187Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:19:17.144816Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:19:32.689970Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:19:34.917469Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:19:41.360831Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-02T10:19:47.226Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 287 79 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "4f98ccf1-0fde-4976-9c01-3d9288c0397e" "maas.apps.e930e379d4db695acbc5.prod.konflux-ocp-ci.dev" "10.134.0.47:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.58:38664 10.134.0.58:443 10.132.0.19:40910 - v1-completions-path 2026-11-02T10:19:49.193380Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:20:03.119790Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:20:13.834965Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:20:17.381158Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:20:29.890608Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:20:32.133441Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:20:32.414579Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-02T10:20:32.404Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 38 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "7fff7591-206d-45be-a70d-aeee0fc9df2d" "maas.apps.e930e379d4db695acbc5.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.58:443 10.132.0.19:42690 - v1-completions-path 2026-11-02T10:20:36.728061Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:20:52.970134Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:21:03.235260Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:21:20.007075Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:21:22.604570Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:23:38.471030Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:23:41.537070Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:25:01.461278Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:25:16.835291Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:25:18.517398Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:25:22.314481Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-02T10:25:22.221Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 65 307 93 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "d60e3c2b-70af-4b6d-8b57-beaa2aa8fa9f" "maas.apps.e930e379d4db695acbc5.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.58:40586 10.134.0.58:443 10.132.0.19:39450 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T10:25:34.200542Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:25:43.758537Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:25:47.684415Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:32:21.298834Z info cache event for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt : REMOVE, pushing to proxy 2026-11-02T10:32:52.783030Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:32:55.699749Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:32:58.531897Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:34:45.906031Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:38:24.326906Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:38:28.088037Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:38:28.261039Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T10:38:28.111Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 72 314 149 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "8792f92f-0c82-42e3-9bc4-bb0beebc8bee" "maas.apps.e930e379d4db695acbc5.prod.konflux-ocp-ci.dev" "10.133.0.136:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.58:48874 10.134.0.58:443 10.132.0.19:41252 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T10:38:32.610Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 52 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "894d364c-1382-4237-9317-ba8ff9af10be" "maas.apps.e930e379d4db695acbc5.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.58:443 10.132.0.19:41260 - v1-completions-path 2026-11-02T10:38:44.485417Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:38:54.863361Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:38:58.221833Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-02T10:39:02.432Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 67 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "25b02d18-7efa-4ba2-afad-70caf5e6b70f" "maas.apps.e930e379d4db695acbc5.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.58:54736 10.134.0.58:443 10.132.0.19:38786 - v1-completions-path 2026-11-02T10:39:15.535237Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:39:17.826754Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:39:21.345122Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-02T10:39:25.658Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 70 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "ccccf738-1bd4-49cc-b30c-451c043b518e" "maas.apps.e930e379d4db695acbc5.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.58:45640 10.134.0.58:443 10.132.0.19:48404 - v1-completions-path [2026-11-02T10:39:29.907Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 58 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "9a9ffa86-59c1-4927-a7ac-9a3000e96b63" "maas.apps.e930e379d4db695acbc5.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.58:45640 10.134.0.58:443 10.132.0.19:41152 - v1-completions-path 2026-11-02T10:39:43.072632Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:39:53.375208Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:39:55.599642Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:39:59.325166Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:40:03.166108Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=3 2026-11-02T10:40:04.844029Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-02T10:40:10.293Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 70 - "52.71.36.72,10.132.0.19" "python-requests/2.33.0" "7ba3dcfd-c92e-48a4-aba7-98e2a1c1910c" "maas.apps.e930e379d4db695acbc5.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.58:45640 10.134.0.58:443 10.132.0.19:46240 - v1-completions-path 2026-11-02T10:40:22.231966Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:40:32.473360Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:40:34.159123Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:40:41.635701Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:40:45.267099Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:41:00.250051Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:41:20.232977Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:41:30.420665Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:41:40.551969Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:41:50.744612Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:42:01.029507Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:42:11.248393Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:42:21.470128Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:42:31.696647Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:42:41.939296Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T10:42:52.196203Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9