2026-11-01T10:04:21.702657Z info FLAG: --concurrency="0" 2026-11-01T10:04:21.702675Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-01T10:04:21.702679Z info FLAG: --help="false" 2026-11-01T10:04:21.702681Z info FLAG: --log_as_json="false" 2026-11-01T10:04:21.702683Z info FLAG: --log_caller="" 2026-11-01T10:04:21.702685Z info FLAG: --log_output_level="default:info" 2026-11-01T10:04:21.702687Z info FLAG: --log_stacktrace_level="default:none" 2026-11-01T10:04:21.702692Z info FLAG: --log_target="[stdout]" 2026-11-01T10:04:21.702694Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-01T10:04:21.702696Z info FLAG: --outlierLogPath="" 2026-11-01T10:04:21.702698Z info FLAG: --profiling="true" 2026-11-01T10:04:21.702700Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-01T10:04:21.702702Z info FLAG: --proxyLogLevel="warning" 2026-11-01T10:04:21.702704Z info FLAG: --serviceCluster="istio-proxy" 2026-11-01T10:04:21.702706Z info FLAG: --stsPort="0" 2026-11-01T10:04:21.702708Z info FLAG: --templateFile="" 2026-11-01T10:04:21.702710Z info FLAG: --tokenManagerPlugin="" 2026-11-01T10:04:21.702712Z info FLAG: --vklog="0" 2026-11-01T10:04:21.702715Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-01T10:04:21.702865Z info Proxy role ips=[10.132.1.115] type=router id=maas-default-gateway-openshift-default-746ff8f49f-6qxcm.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-01T10:04:21.702916Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-01T10:04:21.704304Z info cpu limit detected as 2, setting concurrency 2026-11-01T10:04:21.704504Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-01T10:04:21.704515Z info JWT policy is third-party-jwt 2026-11-01T10:04:21.704517Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-01T10:04:21.706051Z info Starting default Istio SDS Server 2026-11-01T10:04:21.706071Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-01T10:04:21.706087Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-01T10:04:21.706335Z info Opening status port 15020 2026-11-01T10:04:21.707443Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-01T10:04:21.708615Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-01T10:04:21.709403Z info Starting proxy agent 2026-11-01T10:04:21.709433Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-01T10:04:21.718782Z info sds Starting SDS grpc server 2026-11-01T10:04:21.718802Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-01T10:04:21.772191Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-01T10:04:21.865827Z info cache generated new workload certificate resourceName=default latency=158.24441ms ttl=23h59m59.134177964s 2026-11-01T10:04:21.865869Z info cache Root cert has changed, start rotating root cert 2026-11-01T10:04:21.865910Z info cache returned workload trust anchor from cache ttl=23h59m59.134090732s 2026-11-01T10:04:21.919644Z info ads ADS: new connection for node:1 2026-11-01T10:04:21.919701Z info cache returned workload certificate from cache ttl=23h59m59.080300439s 2026-11-01T10:04:21.919911Z info ads ADS: new connection for node:2 2026-11-01T10:04:21.919989Z info cache returned workload trust anchor from cache ttl=23h59m59.080012228s 2026-11-01T10:04:22.000371Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:04:30.875619Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:04:31.818494Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-01T10:04:32.184153Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-01T10:04:33.749373Z info Readiness succeeded in 12.049015246s 2026-11-01T10:04:33.749723Z info Envoy proxy is ready 2026-11-01T10:04:59.935687Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-01T10:05:04.456Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 5518 5791 92 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "b77f3189-8981-42a0-a457-6f3c6b941a5c" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.132.0.31:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.132.1.115:49992 10.132.1.115:443 10.133.0.18:43410 - v1-chat-completions-path 2026-11-01T10:05:19.283762Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:05:23.940695Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:05:27.851187Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-01T10:05:25.087Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 323 2763 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "c8822e1d-5933-495d-9b4e-e3e711c01919" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.133.0.52:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.115:37310 10.132.1.115:443 10.133.0.18:47668 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T10:05:33.966893Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:05:39.105942Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-11-01T10:05:39.011Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 94 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "1677589f-e4e1-40de-b1a1-ba0f9c1dfbee" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.39:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.1.115:34852 10.132.1.115:443 10.133.0.18:49568 - v1-catch-all-path 2026-11-01T10:05:45.347560Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:05:55.039989Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:06:12.709121Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:06:24.377676Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:06:27.063047Z info ads ADS: new connection for node:3 2026-11-01T10:06:27.063275Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-01T10:06:27.063303Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-01T10:06:31.164103Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:07:10.221813Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-01T10:07:17.119Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 333 89 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "9a544d02-c6b8-4fb1-be32-148451173f65" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.132.1.120:8000" outbound|54321||e2e-prio-pool-d609fa82-inference-pool-ip-949aac53.llm.svc.cluster.local 10.132.1.115:47990 10.132.1.115:443 10.133.0.18:33802 - v1-model-routing [2026-11-01T10:07:33.270Z] "POST /v1/chat/completions HTTP/1.1" 403 - - "-" 131 0 39 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "d65a1b6b-546e-40d3-8ec1-4bb3278e1a58" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" outbound|54321||e2e-prio-pool-d609fa82-inference-pool-ip-949aac53.llm.svc.cluster.local - 10.132.1.115:443 10.133.0.18:38346 - v1-model-routing 2026-11-01T10:07:35.513354Z info ads ADS: "" 3 terminated with stream closed 2026-11-01T10:07:35.584853Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:07:41.941663Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:07:54.495493Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:10:31.370971Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:10:58.863571Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:11:29.048010Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:11:42.702395Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:11:48.675492Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:11:53.440446Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T10:11:53.882125Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-01T10:11:53.381Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 58 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "f616ba6c-ccc1-4b06-937e-172df402069d" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.133.0.52:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.115:42372 10.132.1.115:443 10.133.0.18:40892 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T10:12:06.793787Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:12:14.345011Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:12:33.122993Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:13:26.506376Z info ads ADS: new connection for node:4 2026-11-01T10:13:26.510020Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-01T10:13:26.510064Z info cache read certificate from file resource=file-root:system 2026-11-01T10:13:30.434726Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:13:31.216508Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:13:44.200946Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T10:13:44.201211Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-01T10:13:43.982Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1712 219 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "f7584d56-20b1-41cd-926a-7569d931036b" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.1.115:38104 10.132.1.115:443 10.133.0.18:46926 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-01T10:14:23.785029Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-01T10:14:23.785356Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-11-01T10:14:23.570Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1712 214 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "f6e42aca-730b-4f04-a2d6-b21851c06076" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.1.115:38876 10.132.1.115:443 10.133.0.18:56302 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-01T10:14:38.002617Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-01T10:14:38.002911Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-11-01T10:14:37.843Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1712 159 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "7b49d523-3a9d-4169-8c5f-8acb6383beaf" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.1.115:38876 10.132.1.115:443 10.133.0.18:48306 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-01T10:14:49.672054Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-01T10:14:49.672307Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-11-01T10:14:49.533Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1712 138 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "da0ec226-c66c-4670-9fd7-a7e8b6956c09" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.1.115:38876 10.132.1.115:443 10.133.0.18:36450 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-01T10:14:54.422600Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T10:14:54.422857Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-01T10:14:54.201Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1713 221 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "1bb782b2-a2fe-4531-b8a8-c82d3c645aa1" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.1.115:38104 10.132.1.115:443 10.133.0.18:36472 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-01T10:15:06.136321Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T10:15:06.136602Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-01T10:15:06.004Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1722 132 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "05061be5-e2ff-42d3-8937-369b47598ab5" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.1.115:38104 10.132.1.115:443 10.133.0.18:54670 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-01T10:15:31.380519Z info ads ADS: "" 4 terminated 2026-11-01T10:15:31.422553Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:15:33.936078Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:15:44.262669Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:15:48.636645Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=26 [2026-11-01T10:15:48.554Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 82 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "750e165f-cea9-476f-9a6f-5047942f1cf4" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.115:34090 10.132.1.115:443 10.133.0.18:45118 - v1-completions-path 2026-11-01T10:16:03.478299Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:16:09.522507Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:16:14.003316Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-11-01T10:16:13.940Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 63 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "891e9d9b-4821-43a4-a940-9f579494c579" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.1.115:443 10.133.0.18:38062 - v1-completions-path [2026-11-01T10:16:26.551Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 289 78 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "02ac1ce0-9cd3-4c40-9d7c-8408c9c509fc" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.1.115:53922 10.132.1.115:443 10.133.0.18:44792 - v1-completions-path 2026-11-01T10:16:31.350939Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:16:44.754732Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:16:51.741922Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-01T10:16:51.627Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 77 320 114 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "fb49c19a-6f23-4b9f-bfd9-0be9c0c8fc8b" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.133.0.52:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.115:50316 10.132.1.115:443 10.133.0.18:54968 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-01T10:16:53.533Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 70 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "602abe0a-f504-4459-b880-77dde28ba520" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.115:45796 10.132.1.115:443 10.133.0.18:54982 - v1-completions-path [2026-11-01T10:16:55.312Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 60 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "15ca7638-7712-9183-8a9d-3c6c5dfddc70" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.115:45796 10.132.1.115:443 10.133.0.18:52710 - v1-completions-path [2026-11-01T10:16:56.348Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 61 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "847223e5-df1b-41ae-9fd3-4113d5304f13" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.115:34090 10.132.1.115:443 10.133.0.18:52724 - v1-completions-path [2026-11-01T10:16:58.159Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 60 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "29f000d5-d7bb-425c-a91f-1724f4f3ad5c" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.115:45796 10.132.1.115:443 10.133.0.18:52730 - v1-completions-path 2026-11-01T10:17:15.871305Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:17:26.530886Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:17:32.655054Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-01T10:17:45.270Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 64 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "2bbdd3fb-416f-499d-874c-c63830f9a975" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.115:34090 10.132.1.115:443 10.133.0.18:50912 - v1-completions-path 2026-11-01T10:18:01.197701Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:18:05.713478Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:18:18.589516Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:18:27.233179Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:18:36.329524Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:18:44.994823Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:18:54.012412Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:19:11.500403Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:19:17.295524Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:19:36.178554Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:19:41.405025Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-01T10:19:43.274Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 281 70 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "fb7fa54d-8679-42dd-a228-bf4461a59852" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.1.115:41030 10.132.1.115:443 10.133.0.18:36136 - v1-completions-path 2026-11-01T10:19:53.893166Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:20:13.681736Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:20:24.427754Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:20:42.810170Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:20:50.274392Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:22:55.483929Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:24:25.119641Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:24:45.522943Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:24:50.704525Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-01T10:24:51.086Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 90 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "4788c4e2-b24a-4f5b-b7c3-e31576bfec7f" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.1.115:53922 10.132.1.115:443 10.133.0.18:51258 - v1-completions-path 2026-11-01T10:25:04.268495Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:25:18.030821Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:25:23.203985Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-01T10:25:27.401Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 75 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "e3ed6514-04e3-4f00-9d53-52eafad75c83" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.115:34090 10.132.1.115:443 10.133.0.18:51552 - v1-completions-path 2026-11-01T10:25:30.113959Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-01T10:25:29.996Z] "DELETE /v1/api-keys/8ff2d7d7-4951-4a0c-ad89-838591094204 HTTP/1.1" 200 - via_upstream - "-" 0 369 117 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "160d4ffb-693d-459c-94f3-a04585327e69" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.1.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.115:60390 10.132.1.115:443 10.133.0.18:51558 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T10:25:44.072594Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:25:54.753214Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:26:01.335075Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-01T10:26:16.266Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 70 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "e4a3de6f-4614-42b8-ac69-3776871bb02b" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.115:34090 10.132.1.115:443 10.133.0.18:59434 - v1-completions-path 2026-11-01T10:26:31.502353Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:26:38.124508Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:26:44.294913Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-01T10:26:59.066Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 65 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "65b7a16c-4fe6-43a7-8822-242633dbd0ba" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.115:34090 10.132.1.115:443 10.133.0.18:35898 - v1-completions-path [2026-11-01T10:27:01.914Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 57 - "52.71.36.72,10.133.0.18" "python-requests/2.34.2" "8c206ce9-78f2-4905-a6b1-0ad7834f176a" "maas.apps.97301875ced6cd69e07a.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.1.115:443 10.133.0.18:35912 - v1-completions-path 2026-11-01T10:27:16.234871Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:27:26.843952Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:27:31.924028Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:27:38.106916Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:27:49.486055Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:28:12.955276Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:28:23.674690Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:28:28.680443Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:28:46.625394Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:28:52.911323Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:29:16.956722Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:29:36.912825Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:29:47.855527Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:29:58.849915Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:30:09.813261Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:30:20.603658Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:30:31.522877Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:30:42.332233Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:30:53.064482Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:31:03.863230Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-01T10:31:14.512824Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9