2026-11-03T13:17:39.721707Z info FLAG: --concurrency="0" 2026-11-03T13:17:39.721726Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T13:17:39.721730Z info FLAG: --help="false" 2026-11-03T13:17:39.721732Z info FLAG: --log_as_json="false" 2026-11-03T13:17:39.721735Z info FLAG: --log_caller="" 2026-11-03T13:17:39.721737Z info FLAG: --log_output_level="default:info" 2026-11-03T13:17:39.721739Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T13:17:39.721743Z info FLAG: --log_target="[stdout]" 2026-11-03T13:17:39.721745Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T13:17:39.721748Z info FLAG: --outlierLogPath="" 2026-11-03T13:17:39.721750Z info FLAG: --profiling="true" 2026-11-03T13:17:39.721752Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T13:17:39.721754Z info FLAG: --proxyLogLevel="warning" 2026-11-03T13:17:39.721757Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T13:17:39.721759Z info FLAG: --stsPort="0" 2026-11-03T13:17:39.721761Z info FLAG: --templateFile="" 2026-11-03T13:17:39.721763Z info FLAG: --tokenManagerPlugin="" 2026-11-03T13:17:39.721765Z info FLAG: --vklog="0" 2026-11-03T13:17:39.721768Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T13:17:39.721912Z info Proxy role ips=[10.133.0.54] type=router id=maas-default-gateway-openshift-default-84587756dc-mnk7k.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T13:17:39.721963Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T13:17:39.723050Z info cpu limit detected as 2, setting concurrency 2026-11-03T13:17:39.723636Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T13:17:39.723647Z info JWT policy is third-party-jwt 2026-11-03T13:17:39.723650Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T13:17:39.725532Z info Starting default Istio SDS Server 2026-11-03T13:17:39.725568Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T13:17:39.725592Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T13:17:39.726123Z info Opening status port 15020 2026-11-03T13:17:39.727090Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T13:17:39.727556Z info sds Starting SDS grpc server 2026-11-03T13:17:39.727588Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T13:17:39.728715Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T13:17:39.729560Z info Starting proxy agent 2026-11-03T13:17:39.729588Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T13:17:39.790730Z info cache generated new workload certificate resourceName=default latency=62.981217ms ttl=23h59m59.209272593s 2026-11-03T13:17:39.790764Z info cache Root cert has changed, start rotating root cert 2026-11-03T13:17:39.790798Z info cache returned workload trust anchor from cache ttl=23h59m59.209202774s 2026-11-03T13:17:39.792012Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T13:17:39.909556Z info ads ADS: new connection for node:1 2026-11-03T13:17:39.909617Z info cache returned workload certificate from cache ttl=23h59m59.09038399s 2026-11-03T13:17:39.909782Z info ads ADS: new connection for node:2 2026-11-03T13:17:39.909861Z info cache returned workload trust anchor from cache ttl=23h59m59.090141064s 2026-11-03T13:17:39.989674Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:17:48.005242Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:17:48.951553Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T13:17:49.366184Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T13:17:49.965245Z info Readiness succeeded in 10.246500202s 2026-11-03T13:17:49.965570Z info Envoy proxy is ready [2026-11-03T13:17:56.191Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 383 99 - "34.228.250.194,10.132.0.13" "curl/7.76.1" "fd32a88a-a362-4457-8f59-9de70243c62e" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.54:50116 10.133.0.54:443 10.132.0.13:34218 - v1-model-routing [2026-11-03T13:17:56.950Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 88 - "34.228.250.194,10.132.0.13" "curl/7.76.1" "e16b1f40-b0bd-47e5-a1d2-1e0f580c7d9c" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.54:443 10.132.0.13:34222 - v1-model-routing [2026-11-03T13:17:57.711Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 59 - "34.228.250.194,10.132.0.13" "curl/7.76.1" "564bb121-958c-4f9a-909d-2fc343ec612b" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.54:443 10.132.0.13:34224 - v1-model-routing 2026-11-03T13:17:58.015951Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T13:17:58.015977Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T13:17:58.015981Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T13:17:58.015986Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-11-03T13:17:57.997Z] "POST /v1/chat/completions HTTP/1.1" 401 - - "-" 122 88 18 - "34.228.250.194,10.132.0.13" "curl/7.76.1" "df16bdba-6a99-4468-8970-8099c99cda63" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.54:443 10.132.0.13:34232 - v1-model-routing 2026-11-03T13:18:20.392015Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:18:20.392055Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:18:20.392074Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:18:20.392080Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T13:18:20.322Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 69 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "2e528ac1-5797-4efe-a9ce-748cc69fc85d" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.133.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.54:55110 10.133.0.54:443 10.132.0.13:37778 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T13:18:37.053220Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:18:40.268496Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:21:43.670828Z info ads ADS: new connection for node:3 2026-11-03T13:21:43.694822Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-03T13:21:43.694948Z info cache read certificate from file resource=file-root:system [2026-11-03T13:22:51.650Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 113 320 109 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "8efb6901-89df-4918-a7f3-ced5dbb64862" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.54:50116 10.133.0.54:443 10.132.0.13:56358 - v1-chat-completions-path 2026-11-03T13:28:56.486589Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:29:08.722253Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:29:28.049882Z info ads ADS: new connection for node:4 2026-11-03T13:29:28.050111Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T13:29:28.050140Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T13:29:56.653288Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:30:01.274191Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:30:01.274229Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:30:01.274242Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:30:01.274248Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T13:29:59.538Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 66 309 1736 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "180ed8a5-2a87-43a3-9356-b00306db61fb" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.133.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.54:51380 10.133.0.54:443 10.132.0.13:53552 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T13:30:36.164987Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:31:04.421843Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:31:31.835508Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:31:36.549429Z info ads ADS: "" 4 terminated with stream closed 2026-11-03T13:31:55.763048Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:32:00.302449Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:32:00.302488Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:32:00.302499Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T13:32:00.302503Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:32:00.302680Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T13:32:00.128Z] "POST /anything/e2e-rt-sw-2ece70-client HTTP/1.1" 200 - via_upstream - "-" 124 1762 174 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "78a181c1-c568-45a9-9825-ce7663f0d25d" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.54:45320 10.133.0.54:443 10.132.0.13:35426 - llm.external-model-e2e-rt-sw-2ece70-8a582f45b424.0 2026-11-03T13:32:12.276177Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:32:36.858522Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:33:03.063730Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:34:41.950745Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:34:46.628637Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:34:46.628668Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:34:46.628679Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:34:46.628683Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T13:34:46.628824Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-03T13:34:46.628858Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T13:34:46.129Z] "POST /anything/e2e-rt-eq-86abdb-client HTTP/1.1" 200 - via_upstream - "-" 124 1765 499 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "abb1c484-3cac-4691-baff-d07ca7ffa458" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.54:36870 10.133.0.54:443 10.132.0.13:43466 - llm.external-model-e2e-rt-eq-86abdb-c702f89b7e8b.0 2026-11-03T13:35:14.915721Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:35:31.351977Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:35:53.359979Z info ads ADS: "" 3 terminated 2026-11-03T13:36:34.512658Z info ads ADS: new connection for node:5 2026-11-03T13:36:34.524699Z info cache read certificate from file resource=file-root:system 2026-11-03T13:37:18.848667Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:38:08.174624Z info ads ADS: "" 5 terminated 2026-11-03T13:39:04.944756Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:42:23.345056Z info ads ADS: new connection for node:6 2026-11-03T13:42:23.359579Z info cache read certificate from file resource=file-root:system 2026-11-03T13:44:21.170224Z info ads ADS: "" 6 terminated 2026-11-03T13:46:09.460861Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:46:20.470466Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-03T13:46:33.900620Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:48:35.690883Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:48:39.481213Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:48:39.481245Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:48:39.481257Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T13:48:39.481262Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T13:48:39.420Z] "DELETE /v1/api-keys/e2e-trlp-test-key HTTP/1.1" 404 - via_upstream - "-" 0 29 61 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "6c107430-d75d-4439-a542-eca09097ce83" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.133.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.54:60628 10.133.0.54:443 10.132.0.13:49106 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T13:48:50.930010Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:48:52.647417Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:49:02.563648Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:49:05.924495Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:49:09.434753Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:49:24.722824Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:49:44.666112Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:49:54.869948Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:50:05.192183Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:50:15.525154Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:50:25.637607Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:50:35.877695Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:50:46.101984Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:50:56.283891Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:51:06.453952Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:51:16.696091Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:51:54.139402Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:51:58.517588Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-03T13:51:58.437Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 140 405 81 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "db0d648a-f45f-4059-ad94-2728c7fb0dc1" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.132.0.42:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.133.0.54:45702 10.133.0.54:443 10.132.0.13:45666 - v1-chat-completions-path 2026-11-03T13:52:10.047017Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:52:11.736731Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:52:17.513829Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:52:33.062463Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:52:34.639784Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:52:49.203952Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:52:59.949355Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:53:01.738689Z info ads ADS: new connection for node:7 2026-11-03T13:53:01.738853Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T13:53:02.895828Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:53:43.815208Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:54:02.806996Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:54:03.312573Z info ads ADS: "" 7 terminated 2026-11-03T13:54:04.618636Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:54:07.703104Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:54:11.692224Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:54:23.792556Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:54:40.478223Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:54:56.764324Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:55:08.398613Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:55:12.199393Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:55:20.377512Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:55:22.947283Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:55:26.064382Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:55:26.064426Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:55:26.064452Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:55:26.064458Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T13:55:26.001Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 524 63 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "cb2e1a3b-694d-4226-bfb0-64467d6d2d9a" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.133.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.54:53322 10.133.0.54:443 10.132.0.13:36146 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T13:55:26.539Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 69 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "c51b9308-4e80-4932-909d-c44348f9e0cb" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.54:443 10.132.0.13:36152 - v1-completions-path 2026-11-03T13:55:38.587767Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:58:57.148386Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:58:57.148428Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T13:58:57.148452Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T13:58:57.148459Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T13:58:57.074Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 88 330 74 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "cd406d48-c532-44e0-96b1-5e4a029ef33c" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.133.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.54:43122 10.133.0.54:443 10.132.0.13:40302 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T13:58:57.786Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 278 77 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "8f7d6877-d3b1-4904-878c-5e36401a3fb3" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.54:50116 10.133.0.54:443 10.132.0.13:40318 - v1-completions-path [2026-11-03T13:59:11.592Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 65 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "d8dec3eb-e857-4ec0-9d67-7c1337640750" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.54:50116 10.133.0.54:443 10.132.0.13:47776 - v1-completions-path [2026-11-03T13:59:24.902Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 283 66 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "edf021db-b777-42fc-8efe-bc8e1bccea12" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.54:50116 10.133.0.54:443 10.132.0.13:46546 - v1-completions-path [2026-11-03T13:59:25.627Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 295 60 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "48780744-2cde-442b-a794-356e22058f33" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.54:60966 10.133.0.54:443 10.132.0.13:46562 - v1-completions-path 2026-11-03T13:59:40.797319Z info ads ADS: new connection for node:8 2026-11-03T13:59:40.802475Z info cache read certificate from file resource=file-root:system 2026-11-03T13:59:41.476720Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:59:42.909586Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T13:59:59.259478Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:59:59.259515Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:59:59.259533Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T13:59:59.259538Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T13:59:59.259732Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T13:59:59.129Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1716 130 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "7032f319-324f-4def-92ef-a536db9fe08a" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.54:53384 10.133.0.54:443 10.132.0.13:57638 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T14:00:14.596956Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:00:14.596990Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:00:14.597003Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:00:14.597007Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T14:00:14.597175Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T14:00:14.476Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1710 121 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "f2499586-c4b5-4c51-84c3-4eedd8b9751c" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.54:35556 10.133.0.54:443 10.132.0.13:40734 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T14:00:16.768081Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:00:16.768121Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:00:16.768138Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:00:16.768144Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T14:00:16.768371Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T14:00:16.662Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1719 105 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "9ed81da5-e38f-4e6a-b4f7-6cf2de790831" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.54:53384 10.133.0.54:443 10.132.0.13:57678 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T14:00:18.924881Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:00:18.924917Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:00:18.924928Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:00:18.924932Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T14:00:18.925103Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T14:00:18.830Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1716 95 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "c69dd35d-959d-4163-b428-bab13c16a806" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.54:35556 10.133.0.54:443 10.132.0.13:57680 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T14:00:25.421211Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:00:25.421242Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:00:25.421252Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:00:25.421256Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T14:00:25.421425Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T14:00:25.322Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1718 98 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "f2f4bc39-f3a1-46e7-a054-8f171ef34293" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.54:35556 10.133.0.54:443 10.132.0.13:40734 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T14:00:27.575249Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:00:27.575282Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:00:27.575294Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T14:00:27.575298Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:00:27.575482Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T14:00:27.477Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1712 98 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "7b2fd8ea-ad3d-4bed-9bc7-6340caf2b4a2" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.54:53384 10.133.0.54:443 10.132.0.13:40756 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T14:00:49.366274Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:00:49.366310Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:00:49.366322Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:00:49.366326Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T14:00:49.366538Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T14:00:49.269Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1711 97 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "8eb4947f-7918-424d-b406-0e33c125e37b" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.54:35556 10.133.0.54:443 10.132.0.13:35894 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T14:01:00.189302Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:01:00.189337Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:01:00.189376Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:01:00.189381Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T14:01:00.189572Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T14:01:00.093Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1716 96 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "a2f3ac13-6de4-47b5-a58c-d93489bd4d66" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.54:53384 10.133.0.54:443 10.132.0.13:34626 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T14:01:26.681577Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:01:26.808565Z info ads ADS: "" 8 terminated with stream closed 2026-11-03T14:01:28.374837Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:01:31.430574Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T14:01:31.832Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 107 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "b095612d-8dc4-42d0-8fde-1c1b516a54cd" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.54:443 10.132.0.13:47918 - v1-completions-path 2026-11-03T14:01:43.437853Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:01:45.269365Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:01:48.082833Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:01:56.632078Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:02:02.574376Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:02:02.574414Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:02:02.574429Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:02:02.574433Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T14:02:02.507Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 77 320 67 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "7fdd501f-46c4-4f93-8be0-789f40f6b547" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.133.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.54:43728 10.133.0.54:443 10.132.0.13:49776 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T14:02:21.383025Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:02:31.631302Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:02:33.307029Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T14:02:40.663Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 73 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "42a9aa2a-c83e-4820-9f59-0538716a9b99" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.132.0.41:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.54:53820 10.133.0.54:443 10.132.0.13:40980 - v1-completions-path 2026-11-03T14:02:52.303322Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:02:54.017836Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:02:59.872491Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:03:10.795498Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:03:24.465488Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:03:26.055447Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:03:31.712117Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:03:49.816897Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:03:52.442997Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:04:05.204456Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:04:07.303133Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:04:07.619533Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:04:07.619569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T14:04:07.619585Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T14:04:07.619589Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T14:04:07.554Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 89 332 65 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "482b61dd-3afc-4008-a051-60b517d710aa" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.133.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.54:36154 10.133.0.54:443 10.132.0.13:38704 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T14:04:07.679Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 287 63 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "898a847c-0c6b-4c43-81cc-f9a34dad2ec3" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.54:50116 10.133.0.54:443 10.132.0.13:38720 - v1-completions-path [2026-11-03T14:04:07.795Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 37 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "8fa6063d-e5f0-4b02-bc4d-a22b529757dd" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.54:443 10.132.0.13:38724 - v1-completions-path 2026-11-03T14:04:11.794489Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:04:30.161114Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:04:40.398263Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:04:56.894154Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:04:59.826769Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:05:48.357106Z info cache event for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt : REMOVE, pushing to proxy 2026-11-03T14:07:14.442042Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:07:17.479566Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:07:19.673793Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:07:34.812667Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:07:45.026635Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:08:04.977965Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:11:40.687057Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:11:45.112773Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:12:01.284615Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:12:11.432230Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:12:14.750551Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:12:33.409453Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:12:35.525783Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:12:38.817915Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:12:48.125322Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:12:48.125380Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:12:48.125395Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T14:12:48.125400Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T14:12:48.057Z] "DELETE /v1/api-keys/3704cd7d-5086-4978-843d-cd909fbeb0ba HTTP/1.1" 200 - via_upstream - "-" 0 365 68 - "34.228.250.194,10.132.0.13" "python-requests/2.33.0" "fca72918-76cb-4177-878a-6ec4b9bceabd" "maas.apps.013b5c78a9b24bb4ebb6.prod.konflux-ocp-ci.dev" "10.132.0.245:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.54:60590 10.133.0.54:443 10.132.0.13:46308 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T14:12:59.614252Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:13:09.903186Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:13:12.086249Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:13:15.431642Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:13:23.524916Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:13:42.222668Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T14:13:52.399687Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8