2026-11-03T21:14:36.892380Z info FLAG: --concurrency="0" 2026-11-03T21:14:36.892398Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T21:14:36.892403Z info FLAG: --help="false" 2026-11-03T21:14:36.892406Z info FLAG: --log_as_json="false" 2026-11-03T21:14:36.892409Z info FLAG: --log_caller="" 2026-11-03T21:14:36.892413Z info FLAG: --log_output_level="default:info" 2026-11-03T21:14:36.892415Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T21:14:36.892421Z info FLAG: --log_target="[stdout]" 2026-11-03T21:14:36.892425Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T21:14:36.892429Z info FLAG: --outlierLogPath="" 2026-11-03T21:14:36.892432Z info FLAG: --profiling="true" 2026-11-03T21:14:36.892435Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T21:14:36.892437Z info FLAG: --proxyLogLevel="warning" 2026-11-03T21:14:36.892440Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T21:14:36.892443Z info FLAG: --stsPort="0" 2026-11-03T21:14:36.892446Z info FLAG: --templateFile="" 2026-11-03T21:14:36.892449Z info FLAG: --tokenManagerPlugin="" 2026-11-03T21:14:36.892453Z info FLAG: --vklog="0" 2026-11-03T21:14:36.892456Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T21:14:36.892668Z info Proxy role ips=[10.132.0.153] type=router id=maas-default-gateway-openshift-default-57f78bcd55-4f66q.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T21:14:36.892725Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T21:14:36.893892Z info cpu limit detected as 2, setting concurrency 2026-11-03T21:14:36.894206Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T21:14:36.894218Z info JWT policy is third-party-jwt 2026-11-03T21:14:36.894222Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T21:14:36.896117Z info Starting default Istio SDS Server 2026-11-03T21:14:36.896283Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T21:14:36.896311Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T21:14:36.896517Z info Opening status port 15020 2026-11-03T21:14:36.897384Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T21:14:36.898368Z info sds Starting SDS grpc server 2026-11-03T21:14:36.898384Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T21:14:36.898929Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T21:14:36.899776Z info Starting proxy agent 2026-11-03T21:14:36.899805Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T21:14:36.963380Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T21:14:37.036249Z info cache generated new workload certificate resourceName=default latency=137.716159ms ttl=23h59m59.963755097s 2026-11-03T21:14:37.036287Z info cache Root cert has changed, start rotating root cert 2026-11-03T21:14:37.036325Z info cache returned workload trust anchor from cache ttl=23h59m59.963675345s 2026-11-03T21:14:37.118533Z info ads ADS: new connection for node:1 2026-11-03T21:14:37.118630Z info cache returned workload certificate from cache ttl=23h59m59.881373264s 2026-11-03T21:14:37.119208Z info ads ADS: new connection for node:2 2026-11-03T21:14:37.119308Z info cache returned workload trust anchor from cache ttl=23h59m59.880692926s 2026-11-03T21:14:37.204583Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:14:45.325851Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:14:46.145257Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T21:14:46.373756Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T21:14:47.364199Z info Readiness succeeded in 10.474215152s 2026-11-03T21:14:47.364498Z info Envoy proxy is ready 2026-11-03T21:15:19.374855Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:15:39.280980Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:15:44.314958Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:15:48.049278Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-11-03T21:15:47.961Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 91 3616 87 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "88f514cb-3f23-4a5a-b992-6acc28c23359" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.38:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.153:37594 10.132.0.153:443 10.132.0.11:40122 - v1-catch-all-model-routing 2026-11-03T21:15:53.451547Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:15:56.514635Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:15:56.514671Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:15:56.514684Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T21:15:56.514689Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T21:15:55.845Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 317 669 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "431c71b9-717f-42da-a8e6-c1ce462e30d2" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.153:57988 10.132.0.153:443 10.132.0.11:40150 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:16:08.202604Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:16:16.126771Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:16:33.492778Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:16:45.237025Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:16:47.646552Z info ads ADS: new connection for node:3 2026-11-03T21:16:47.646723Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T21:16:47.646759Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T21:16:52.454241Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:17:31.370490Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:17:34.714643Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:17:34.714676Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:17:34.714689Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T21:17:34.714694Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T21:17:33.964Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 313 750 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "515a2556-16e7-4848-af1c-0ed1248bf545" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.153:38328 10.132.0.153:443 10.132.0.11:37224 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T21:17:35.912Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 342 93 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "a7d0391c-a546-4e93-bd0f-86175c9c5124" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.133.0.247:8000" outbound|54321||e2e-prio-pool-9f7f5f7b-inference-pool-ip-ca181a17.llm.svc.cluster.local 10.132.0.153:54438 10.132.0.153:443 10.132.0.11:40564 - v1-model-routing [2026-11-03T21:17:40.476Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 348 62 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "e31a4da8-6456-4410-9ce4-40a8d9bf29d2" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.133.0.247:8000" outbound|54321||e2e-prio-pool-9f7f5f7b-inference-pool-ip-ca181a17.llm.svc.cluster.local 10.132.0.153:54440 10.132.0.153:443 10.132.0.11:40578 - v1-model-routing 2026-11-03T21:17:58.080014Z info ads ADS: "" 3 terminated 2026-11-03T21:17:58.343154Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:18:03.496573Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:18:17.306034Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:21:06.149441Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:21:06.149475Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:21:06.149489Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:21:06.149495Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T21:21:06.084Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 65 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "e35db5e5-554a-4cb5-9877-5b8e1f488b0d" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.132.0.47:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.153:35130 10.132.0.153:443 10.132.0.11:36798 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:21:16.157786Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:21:41.997231Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:22:08.277730Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:22:22.990963Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:22:34.994160Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:22:47.851453Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:22:53.191950Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:22:57.136687Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-03T21:22:57.079Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 75 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "0379d47e-81a4-4aa3-8aee-3382d2ff3aa5" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.153:52906 10.132.0.153:443 10.132.0.11:39708 - v1-completions-path [2026-11-03T21:23:00.469Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 55 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "6974b63b-7f4d-4f5a-8f8b-618fd37d3536" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.153:443 10.132.0.11:39734 - v1-completions-path 2026-11-03T21:23:14.863987Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:24:03.686642Z info ads ADS: new connection for node:4 2026-11-03T21:24:03.691024Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-03T21:24:03.691110Z info cache read certificate from file resource=file-root:system 2026-11-03T21:24:07.730503Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:24:08.885581Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:24:12.286673Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"196","StreamId":"13661517471183984044"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=24 [2026-11-03T21:24:12.219Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 67 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "5809024e-24a5-4e0c-97be-80f0e60fe16f" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.132.0.153:443 10.132.0.11:58682 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-11-03T21:24:59.889594Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:24:59.889627Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:24:59.889637Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T21:24:59.889641Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:24:59.889820Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T21:24:59.687Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1534 202 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "0e86453b-168c-428a-b35e-9da4381bf380" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.153:54034 10.132.0.153:443 10.132.0.11:46100 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T21:25:50.557522Z info ads ADS: "" 4 terminated 2026-11-03T21:25:50.614491Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:25:53.649887Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:26:04.871311Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:26:23.853609Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:26:28.803308Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:26:53.625972Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:27:06.685896Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:27:17.616Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 70 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "2e2295a4-13a9-4781-9c8f-14ccdb6df511" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.153:52906 10.132.0.153:443 10.132.0.11:51826 - v1-completions-path [2026-11-03T21:27:19.023Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 59 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "03d747b0-bf79-48f3-9203-652f8ff430ab" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.153:52906 10.132.0.153:443 10.132.0.11:51832 - v1-completions-path 2026-11-03T21:27:20.734944Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-11-03T21:27:20.677Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 289 65 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "bedf899e-621f-4784-ad76-4f22ef9d673e" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.153:59496 10.132.0.153:443 10.132.0.11:51836 - v1-completions-path 2026-11-03T21:27:37.798398Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:27:49.099306Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:27:54.098839Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:28:07.356Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 63 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "c2f80f97-1fd1-46cb-9245-36f3384b798b" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.153:52906 10.132.0.153:443 10.132.0.11:51052 - v1-completions-path 2026-11-03T21:28:23.559947Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:28:28.404014Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:28:31.741Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 74 291 81 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "681aaaf9-5ed8-40dc-b8e1-c80614622055" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.133.0.33:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.132.0.153:44596 10.132.0.153:443 10.132.0.11:47904 - v1-completions-path 2026-11-03T21:28:40.995802Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:28:51.521622Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:29:02.726567Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:29:09.668209Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:29:24.981073Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:29:38.279608Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:29:44.974962Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:30:03.701132Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:30:09.598814Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:30:21.491542Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:30:47.048063Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:30:58.174599Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:31:18.524412Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:31:26.908019Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:33:52.048457Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:33:57.271506Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:35:27.388222Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:35:45.151423Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:35:50.149678Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:36:02.753668Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:36:19.613008Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:36:23.446464Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:36:43.237467Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:36:53.936188Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:37:00.838379Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:37:14.500Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 66 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "1524a13c-bb0c-45dd-8e42-5b07d1bd539f" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.153:59496 10.132.0.153:443 10.132.0.11:35082 - v1-completions-path 2026-11-03T21:37:29.591355Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:37:36.431198Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:37:43.081685Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:37:47.470525Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:37:47.470560Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:37:47.470573Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:37:47.470578Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T21:37:47.360Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 68 310 109 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "a0fa499c-b7e0-46a4-9168-1e6f4789fa51" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.133.1.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.153:54272 10.132.0.153:443 10.132.0.11:44866 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T21:37:51.082Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 63 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "616d8c7f-8459-4a57-bb3a-6da1f6c67ec4" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.153:52906 10.132.0.153:443 10.132.0.11:44884 - v1-completions-path [2026-11-03T21:37:53.681Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 64 - "44.212.242.249,10.132.0.11" "python-requests/2.34.2" "5946e74b-23a0-4b2f-ac77-0f6a0a6f3276" "maas.apps.efe410098b32fc21bf24.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.153:52906 10.132.0.153:443 10.132.0.11:44898 - v1-completions-path 2026-11-03T21:38:11.531555Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:38:22.194891Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:38:29.145044Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:38:35.933278Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:38:46.960410Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:39:10.341671Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:39:23.471725Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:39:33.838382Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:39:40.677969Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:40:03.573417Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:40:25.570260Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:40:47.687074Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:41:07.342423Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:41:18.062133Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:41:28.691048Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:41:39.601706Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:41:50.434379Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:42:00.987863Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:42:17.707131Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2