2026-11-02T23:59:39.953026Z info FLAG: --concurrency="0" 2026-11-02T23:59:39.953055Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-02T23:59:39.953060Z info FLAG: --help="false" 2026-11-02T23:59:39.953062Z info FLAG: --log_as_json="false" 2026-11-02T23:59:39.953064Z info FLAG: --log_caller="" 2026-11-02T23:59:39.953066Z info FLAG: --log_output_level="default:info" 2026-11-02T23:59:39.953069Z info FLAG: --log_stacktrace_level="default:none" 2026-11-02T23:59:39.953126Z info FLAG: --log_target="[stdout]" 2026-11-02T23:59:39.953129Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-02T23:59:39.953130Z info FLAG: --outlierLogPath="" 2026-11-02T23:59:39.953133Z info FLAG: --profiling="true" 2026-11-02T23:59:39.953135Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-02T23:59:39.953137Z info FLAG: --proxyLogLevel="warning" 2026-11-02T23:59:39.953138Z info FLAG: --serviceCluster="istio-proxy" 2026-11-02T23:59:39.953141Z info FLAG: --stsPort="0" 2026-11-02T23:59:39.953143Z info FLAG: --templateFile="" 2026-11-02T23:59:39.953144Z info FLAG: --tokenManagerPlugin="" 2026-11-02T23:59:39.953147Z info FLAG: --vklog="0" 2026-11-02T23:59:39.953150Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-02T23:59:39.953296Z info Proxy role ips=[10.133.0.182] type=router id=maas-default-gateway-openshift-default-77bfb58f8c-lgl4w.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-02T23:59:39.953336Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-02T23:59:39.954693Z info cpu limit detected as 2, setting concurrency 2026-11-02T23:59:39.955499Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-02T23:59:39.955538Z info JWT policy is third-party-jwt 2026-11-02T23:59:39.955542Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-02T23:59:39.957876Z info Starting default Istio SDS Server 2026-11-02T23:59:39.957864Z info Opening status port 15020 2026-11-02T23:59:39.957894Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-02T23:59:39.957917Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-02T23:59:39.958633Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-02T23:59:39.959247Z info sds Starting SDS grpc server 2026-11-02T23:59:39.959269Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-02T23:59:39.960326Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-02T23:59:39.961128Z info Starting proxy agent 2026-11-02T23:59:39.961159Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-02T23:59:40.028228Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-02T23:59:40.131268Z info cache generated new workload certificate resourceName=default latency=171.855866ms ttl=23h59m59.868737922s 2026-11-02T23:59:40.131399Z info cache Root cert has changed, start rotating root cert 2026-11-02T23:59:40.131475Z info cache returned workload trust anchor from cache ttl=23h59m59.868525918s 2026-11-02T23:59:40.274807Z info ads ADS: new connection for node:1 2026-11-02T23:59:40.278485Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-02T23:59:40.278531Z info cache read certificate from file resource=file-root:system 2026-11-02T23:59:40.309033Z info ads ADS: new connection for node:2 2026-11-02T23:59:40.309111Z info cache returned workload certificate from cache ttl=23h59m59.690894045s 2026-11-02T23:59:40.310345Z info ads ADS: new connection for node:3 2026-11-02T23:59:40.310411Z info cache returned workload trust anchor from cache ttl=23h59m59.689590243s 2026-11-02T23:59:40.421624Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-02T23:59:48.714126Z info Readiness succeeded in 8.763478207s 2026-11-02T23:59:48.714538Z info Envoy proxy is ready 2026-11-03T00:00:00.284476Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T00:00:11.963Z] "POST /llm/e2e-dft-66da83/anything/e2e-dft-66da83-cbafab HTTP/1.1" 503 - - "-" 109 0 184 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "6f8e3c3f-ebf1-40bf-a585-8cae1e730e04" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-66da83.llm.svc.cluster.local - 10.133.0.182:443 10.132.0.10:40368 - llm.external-model-e2e-dft-66da83-99ca68864c21.2 [2026-11-03T00:01:21.097Z] "POST /llm/e2e-dft-66da83/anything/e2e-dft-66da83-cbafab HTTP/1.1" 503 - - "-" 109 0 78 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "30c128c6-6d17-4116-b9e2-d505549b792c" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-66da83.llm.svc.cluster.local - 10.133.0.182:443 10.132.0.10:41042 - llm.external-model-e2e-dft-66da83-99ca68864c21.2 [2026-11-03T00:01:36.920Z] "POST /llm/e2e-dft-66da83/anything/e2e-dft-66da83-cbafab HTTP/1.1" 503 - - "-" 109 0 68 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "68f0d0ae-2ef9-41ac-a1bf-61f8357afe77" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-66da83.llm.svc.cluster.local - 10.133.0.182:443 10.132.0.10:35994 - llm.external-model-e2e-dft-66da83-99ca68864c21.2 [2026-11-03T00:01:58.183Z] "POST /llm/e2e-dft-66da83/anything/e2e-dft-66da83-cbafab HTTP/1.1" 503 - - "-" 109 0 72 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "928cbf50-ce0f-4b02-8468-c5afd020c37c" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-66da83.llm.svc.cluster.local - 10.133.0.182:443 10.132.0.10:53982 - llm.external-model-e2e-dft-66da83-99ca68864c21.2 [2026-11-03T00:02:14.139Z] "POST /llm/e2e-dft-66da83/anything/e2e-dft-66da83-cbafab HTTP/1.1" 503 - - "-" 109 0 66 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "37b81b80-7151-40a7-9798-3c5cef6dfb46" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-66da83.llm.svc.cluster.local - 10.133.0.182:443 10.132.0.10:42020 - llm.external-model-e2e-dft-66da83-99ca68864c21.2 [2026-11-03T00:03:17.861Z] "POST /llm/e2e-dft-66da83/anything/e2e-dft-66da83-cbafab HTTP/1.1" 503 - - "-" 109 0 69 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "d9b35b69-4954-48fd-945a-d473dd01a1e0" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-66da83.llm.svc.cluster.local - 10.133.0.182:443 10.132.0.10:47290 - llm.external-model-e2e-dft-66da83-99ca68864c21.2 [2026-11-03T00:03:33.666Z] "POST /llm/e2e-dft-66da83/anything/e2e-dft-66da83-cbafab HTTP/1.1" 503 - - "-" 109 0 71 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "a4106097-d6ab-4c4f-a136-bc077a7e41b4" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-66da83.llm.svc.cluster.local - 10.133.0.182:443 10.132.0.10:45098 - llm.external-model-e2e-dft-66da83-99ca68864c21.2 2026-11-03T00:03:54.931592Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-03T00:03:54.870Z] "POST /llm/e2e-dft-66da83/anything/e2e-dft-66da83-cbafab HTTP/1.1" 503 - - "-" 109 0 65 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "0fac81b6-27d0-4056-b885-4d1b49a58772" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-66da83.llm.svc.cluster.local - 10.133.0.182:443 10.132.0.10:37284 - llm.external-model-e2e-dft-66da83-99ca68864c21.2 2026-11-03T00:04:53.808814Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:04:57.360712Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:04:57.408227Z info ads ADS: "" 1 terminated 2026-11-03T00:04:59.205117Z info ads ADS: new connection for node:4 2026-11-03T00:04:59.211672Z info cache read certificate from file resource=file-root:system 2026-11-03T00:05:03.534516Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:05:04.750958Z info ads ADS: "" 4 terminated with stream closed 2026-11-03T00:05:05.908633Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:05:11.307244Z info ads ADS: new connection for node:5 2026-11-03T00:05:11.312007Z info cache read certificate from file resource=file-root:system 2026-11-03T00:05:14.319058Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:05:17.643829Z info ads ADS: "" 5 terminated with stream closed 2026-11-03T00:05:18.228945Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:05:24.766575Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:05:57.581304Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:06:58.846382Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:07:28.946224Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:07:45.015633Z info ads ADS: new connection for node:6 2026-11-03T00:07:45.020881Z info cache read certificate from file resource=file-root:system 2026-11-03T00:07:49.532616Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:07:51.169602Z info ads ADS: "" 6 terminated with stream closed 2026-11-03T00:07:53.061243Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:08:10.566773Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:08:53.038291Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:21:38.934728Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:21:38.934771Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:21:38.934790Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T00:21:38.934795Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T00:21:38.750Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 30 315 184 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "78181ef7-11c9-4d49-bf9b-6abb6b036422" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "10.133.0.27:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.182:52026 10.133.0.182:443 10.132.0.10:52154 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T00:21:43.543458Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:21:43.543494Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:21:43.543510Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:21:43.543515Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T00:21:43.434Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 109 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "25ff96ab-dc7e-4960-8f3c-250c42655540" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "10.133.0.27:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.182:52026 10.133.0.182:443 10.132.0.10:52170 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T00:21:47.439393Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:22:13.196162Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:22:47.628183Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T00:22:50.285Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 147 412 82 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "d67efc50-9525-4524-bfee-4b712231c223" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "10.133.0.38:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.133.0.182:47104 10.133.0.182:443 10.132.0.10:38164 - v1-chat-completions-path 2026-11-03T00:23:09.734582Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:23:12.157649Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:23:12.157686Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:23:12.157700Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:23:12.157705Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T00:23:12.044Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 323 113 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "ae60fdf8-bd65-4ed6-8247-cfd24eeb5f44" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "10.133.0.27:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.182:55784 10.133.0.182:443 10.132.0.10:36246 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T00:23:17.573178Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:23:21.032941Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 2026-11-03T00:23:21.497686Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-03T00:23:21.501642Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-11-03T00:23:20.905Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 127 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "9d423bcc-d61c-4509-b1f2-1d95232f3bee" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "10.133.0.36:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.182:39598 10.133.0.182:443 10.132.0.10:52218 - v1-catch-all-path [2026-11-03T00:23:21.332Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 171 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "f0d14599-dafc-4e77-8cf1-fe929f0cad1b" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "10.133.0.36:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.182:39614 10.133.0.182:443 10.132.0.10:52232 - v1-catch-all-path [2026-11-03T00:23:24.720Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 429 - - "-" 61 18 115 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "9512e071-2e4a-4335-b3b6-5918b08d121e" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.182:443 10.132.0.10:52260 - v1-catch-all-path 2026-11-03T00:23:32.995081Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:23:42.979070Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:23:59.726065Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:24:11.374897Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:24:14.776476Z info ads ADS: new connection for node:7 2026-11-03T00:24:14.776729Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T00:24:14.776769Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T00:24:18.631091Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:24:57.652240Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:25:22.006482Z info ads ADS: "" 7 terminated with stream closed 2026-11-03T00:25:22.451148Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:25:27.411082Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:25:43.319702Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:28:17.455759Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:28:43.638857Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:29:06.779370Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:29:06.779405Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:29:06.779422Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:29:06.779427Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T00:29:06.660Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 80 323 119 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "7ac23f16-accc-4801-b91e-587e54df2b8f" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "10.133.0.27:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.182:54006 10.133.0.182:443 10.132.0.10:47372 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T00:29:07.034111Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:29:07.034164Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:29:07.034193Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:29:07.034203Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T00:29:06.972Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 628 61 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "7f043ee4-c84f-44b6-ab71-cb4b4818ff7c" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "10.133.0.27:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.182:54006 10.133.0.182:443 10.132.0.10:47376 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T00:29:09.612495Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:29:23.508283Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:29:27.021779Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:29:27.021816Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:29:27.021830Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T00:29:27.021834Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T00:29:26.968Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 52 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "402b99ee-7e91-46df-8d77-c2a5a6e64b64" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "10.133.0.27:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.182:54006 10.133.0.182:443 10.132.0.10:59058 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T00:29:30.474540Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:29:41.148087Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:29:46.217815Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:30:04.637537Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:30:52.919585Z info ads ADS: new connection for node:8 2026-11-03T00:30:52.924746Z info cache read certificate from file resource=file-root:system 2026-11-03T00:30:56.550601Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:30:58.112039Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:31:01.154793Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"536","StreamId":"11799282838609066836"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=25 [2026-11-03T00:31:01.090Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 64 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "5e27e063-a0ed-47a3-a16a-1d4aed3f91fa" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.133.0.182:443 10.132.0.10:56584 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-11-03T00:31:07.960085Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"541","StreamId":"5720311612933102661"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=24 [2026-11-03T00:31:07.894Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 66 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "ef13891e-3f1f-4c93-8157-bbe8a244d1bd" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.133.0.182:443 10.132.0.10:53924 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-11-03T00:31:12.684797Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:31:12.684843Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:31:12.684860Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:31:12.684866Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T00:31:12.685096Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T00:31:12.490Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1716 194 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "c60499f2-ee2f-48d0-8c71-41376da21e2f" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.182:46868 10.133.0.182:443 10.132.0.10:53940 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T00:31:31.733936Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:31:31.733974Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:31:31.734010Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:31:31.734016Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T00:31:31.734220Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T00:31:31.607Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1716 126 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "3563d74f-ff2a-4a99-9164-cb78903119fb" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.182:46868 10.133.0.182:443 10.132.0.10:58848 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T00:31:49.399403Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-03T00:32:09.594073Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:32:09.594108Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:32:09.594122Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:32:09.594128Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T00:32:09.594327Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T00:32:09.398Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1723 195 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "a6af331a-660b-4930-beae-89123a1e18e6" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.182:35706 10.133.0.182:443 10.132.0.10:37368 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T00:32:25.933574Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:32:25.933604Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:32:25.933613Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:32:25.933618Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T00:32:25.933777Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T00:32:25.808Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1721 125 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "5338c3f4-3117-4da7-ad83-fe103975f40b" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.182:46868 10.133.0.182:443 10.132.0.10:39634 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T00:32:30.596705Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:32:30.596738Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:32:30.596748Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:32:30.596752Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T00:32:30.596910Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T00:32:30.470Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1714 126 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "574f389d-34ea-4de6-b0f6-8648adc01a16" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.182:35706 10.133.0.182:443 10.132.0.10:33310 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T00:32:50.439415Z info ads ADS: "" 8 terminated 2026-11-03T00:32:50.482626Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:32:53.312154Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:33:04.239795Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T00:33:08.519Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 75 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "a36d824f-a5e1-4618-b8d5-6efd26dbed40" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.182:44754 10.133.0.182:443 10.132.0.10:42060 - v1-completions-path 2026-11-03T00:33:22.982407Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:33:28.019039Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T00:33:31.640Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 57 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "9c988ced-b74a-4f46-a8df-ae7d8eda922e" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.182:44754 10.133.0.182:443 10.132.0.10:40500 - v1-completions-path [2026-11-03T00:33:32.955Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 63 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "62289478-f7ca-441e-91a6-9fff5a8fedfb" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.182:54932 10.133.0.182:443 10.132.0.10:40522 - v1-completions-path 2026-11-03T00:33:47.821654Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:34:06.703244Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:34:35.641005Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:34:46.429019Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:34:51.252477Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T00:35:04.828Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 62 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "a5847bf5-fc30-4968-b00a-748f48573d06" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.182:54932 10.133.0.182:443 10.132.0.10:44502 - v1-completions-path [2026-11-03T00:35:06.886Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 55 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "57f64221-d515-4abb-8366-2a1c5281ee10" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.182:44754 10.133.0.182:443 10.132.0.10:44506 - v1-completions-path [2026-11-03T00:35:07.392Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 57 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "d36c714b-ad51-4db4-a718-87e35e0a3429" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.182:44754 10.133.0.182:443 10.132.0.10:44520 - v1-completions-path 2026-11-03T00:35:21.375365Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:35:26.317820Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T00:35:29.536Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 74 290 76 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "a7eb8e64-91e9-4f6c-9abc-cb8207102bf2" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "10.133.0.39:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.133.0.182:56370 10.133.0.182:443 10.132.0.10:45698 - v1-completions-path 2026-11-03T00:35:40.019821Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:35:49.310497Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:35:57.263707Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:36:06.731594Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:36:16.399540Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:36:35.050241Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:36:43.163559Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:36:53.577264Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:37:06.495225Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:37:29.465919Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:37:40.055267Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:41:59.138200Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:42:16.329944Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:42:20.784585Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:42:23.654015Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:42:23.654060Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:42:23.654081Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:42:23.654089Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T00:42:23.520Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 65 308 133 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "ad2cfc49-f680-42cd-9a65-26afaf0a273b" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "10.133.0.27:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.182:40224 10.133.0.182:443 10.132.0.10:34726 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T00:42:33.361864Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:42:50.471118Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:42:55.141302Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:43:15.171461Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:43:25.742374Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:43:31.813115Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T00:43:44.915Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 65 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "a014de00-62a6-460b-b405-14528ea658fb" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.182:54932 10.133.0.182:443 10.132.0.10:58564 - v1-completions-path 2026-11-03T00:43:58.997510Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:44:04.970296Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:44:10.668176Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T00:44:18.560Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 63 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "30d21dbc-9473-4a92-94c4-237b3540e647" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.182:54932 10.133.0.182:443 10.132.0.10:33786 - v1-completions-path [2026-11-03T00:44:23.091Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 68 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "c208089a-3874-47ba-914c-bcacb0a127ee" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.182:54932 10.133.0.182:443 10.132.0.10:33796 - v1-completions-path 2026-11-03T00:44:41.821334Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:44:52.519024Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:44:58.897782Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:45:04.823821Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:45:17.041077Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-03T00:45:26.524Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 56 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "4b17b5bb-9c42-4932-8cdd-586bb7c2b01f" "maas.apps.9bf2ce48968e6843c10f.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.182:443 10.132.0.10:50970 - v1-completions-path 2026-11-03T00:45:40.369742Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:45:51.394122Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:45:56.295210Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:46:13.602093Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:46:23.470032Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:46:44.028305Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:47:06.024015Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:47:28.078702Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:47:48.071543Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:47:59.119413Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:47:59.657124Z info cache event for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt : REMOVE, pushing to proxy 2026-11-03T00:48:09.761308Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:48:20.536325Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:48:31.382247Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-03T00:48:42.041666Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9