2026-11-03T04:29:08.871276Z info FLAG: --concurrency="0" 2026-11-03T04:29:08.871306Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T04:29:08.871310Z info FLAG: --help="false" 2026-11-03T04:29:08.871312Z info FLAG: --log_as_json="false" 2026-11-03T04:29:08.871314Z info FLAG: --log_caller="" 2026-11-03T04:29:08.871316Z info FLAG: --log_output_level="default:info" 2026-11-03T04:29:08.871318Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T04:29:08.871323Z info FLAG: --log_target="[stdout]" 2026-11-03T04:29:08.871325Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T04:29:08.871327Z info FLAG: --outlierLogPath="" 2026-11-03T04:29:08.871329Z info FLAG: --profiling="true" 2026-11-03T04:29:08.871331Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T04:29:08.871333Z info FLAG: --proxyLogLevel="warning" 2026-11-03T04:29:08.871335Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T04:29:08.871338Z info FLAG: --stsPort="0" 2026-11-03T04:29:08.871339Z info FLAG: --templateFile="" 2026-11-03T04:29:08.871341Z info FLAG: --tokenManagerPlugin="" 2026-11-03T04:29:08.871344Z info FLAG: --vklog="0" 2026-11-03T04:29:08.871346Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T04:29:08.871530Z info Proxy role ips=[10.132.0.144] type=router id=maas-default-gateway-openshift-default-76f4876667-rnx7l.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T04:29:08.871579Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T04:29:08.872741Z info cpu limit detected as 2, setting concurrency 2026-11-03T04:29:08.872943Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T04:29:08.872955Z info JWT policy is third-party-jwt 2026-11-03T04:29:08.872958Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T04:29:08.874912Z info Starting default Istio SDS Server 2026-11-03T04:29:08.874934Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T04:29:08.874957Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T04:29:08.875276Z info Opening status port 15020 2026-11-03T04:29:08.876188Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T04:29:08.877594Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T04:29:08.878493Z info Starting proxy agent 2026-11-03T04:29:08.878533Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T04:29:08.887221Z info sds Starting SDS grpc server 2026-11-03T04:29:08.887254Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T04:29:08.942380Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T04:29:08.956689Z info cache generated new workload certificate resourceName=default latency=80.692604ms ttl=23h59m59.04331611s 2026-11-03T04:29:08.956724Z info cache Root cert has changed, start rotating root cert 2026-11-03T04:29:08.956761Z info cache returned workload trust anchor from cache ttl=23h59m59.043239427s 2026-11-03T04:29:09.090224Z info ads ADS: new connection for node:1 2026-11-03T04:29:09.090381Z info cache returned workload certificate from cache ttl=23h59m58.90962169s 2026-11-03T04:29:09.091015Z info ads ADS: new connection for node:2 2026-11-03T04:29:09.091093Z info cache returned workload trust anchor from cache ttl=23h59m58.908908977s 2026-11-03T04:29:09.176657Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:29:17.342586Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:29:18.365701Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T04:29:19.452335Z info Readiness succeeded in 10.583939506s 2026-11-03T04:29:19.452937Z info Envoy proxy is ready 2026-11-03T04:29:19.951249Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:29:29.757775Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:29:34.811153Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-11-03T04:29:34.720Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 90 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "24ffea55-07be-4efe-8c0e-e1d06bf12df4" "maas.apps.bbf32bbb4c5c76878149.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.144:50174 10.132.0.144:443 10.132.0.10:38270 - v1-catch-all-path 2026-11-03T04:29:43.965069Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:29:53.447594Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:30:10.718592Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:30:22.570506Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:30:26.588468Z info ads ADS: new connection for node:3 2026-11-03T04:30:26.588831Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T04:30:26.588913Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T04:30:30.171322Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:31:13.984623Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-11-03T04:31:18.757Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 335 98 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "786efc0f-b7db-4156-ac4d-30a102fae749" "maas.apps.bbf32bbb4c5c76878149.prod.konflux-ocp-ci.dev" "10.134.0.232:8000" outbound|54321||e2e-prio-pool-edfabb23-inference-pool-ip-ca68813e.llm.svc.cluster.local 10.132.0.144:54502 10.132.0.144:443 10.132.0.10:49510 - v1-model-routing [2026-11-03T04:31:23.498Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 346 61 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "6541ddca-f8da-4b74-b125-dacd445390f9" "maas.apps.bbf32bbb4c5c76878149.prod.konflux-ocp-ci.dev" "10.134.0.232:8000" outbound|54321||e2e-prio-pool-edfabb23-inference-pool-ip-ca68813e.llm.svc.cluster.local 10.132.0.144:54502 10.132.0.144:443 10.132.0.10:51262 - v1-model-routing 2026-11-03T04:31:41.108106Z info ads ADS: "" 3 terminated with stream closed 2026-11-03T04:31:41.464586Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:31:46.153939Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-11-03T04:31:46.165671Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T04:31:46.165710Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T04:31:46.165729Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T04:31:46.165734Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T04:31:46.470876Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-11-03T04:31:46.039Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 90 332 126 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "b88b1631-b74b-437b-943c-765d437be24a" "maas.apps.bbf32bbb4c5c76878149.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.144:53114 10.132.0.144:443 10.132.0.10:35958 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T04:32:04.328063Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:35:01.043433Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:35:27.121766Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:35:51.783382Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:36:07.848420Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:36:16.996698Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:36:23.904293Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:36:33.830408Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:36:38.566786Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:36:58.496530Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:37:47.612687Z info ads ADS: new connection for node:4 2026-11-03T04:37:47.616742Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-03T04:37:47.616801Z info cache read certificate from file resource=file-root:system 2026-11-03T04:37:51.514274Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:37:53.000644Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:38:15.016839Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T04:38:15.016873Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T04:38:15.016882Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T04:38:15.016886Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T04:38:15.017038Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T04:38:14.888Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1713 128 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "1e1a2097-5ecc-474e-bdc8-f58c8dbce68b" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.144:59844 10.132.0.144:443 10.132.0.10:39326 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T04:38:56.886915Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T04:38:56.886953Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T04:38:56.886967Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T04:38:56.886972Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T04:38:56.887160Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T04:38:56.778Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1712 109 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "c7337c56-566e-449c-9a53-8250656985a4" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.144:59844 10.132.0.144:443 10.132.0.10:43756 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T04:39:28.620601Z info ads ADS: "" 4 terminated 2026-11-03T04:39:28.665666Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:39:31.330876Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:39:42.592798Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-11-03T04:39:46.035Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 83 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "04ad8ac7-bc97-4ffe-aea0-750a37126209" "maas.apps.bbf32bbb4c5c76878149.prod.konflux-ocp-ci.dev" "10.133.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.144:58054 10.132.0.144:443 10.132.0.10:33848 - v1-completions-path 2026-11-03T04:40:01.565725Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:40:06.455317Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-11-03T04:40:12.395Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 299 75 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "5d2c1bc9-a49d-401c-a846-42d25bfe43f9" "maas.apps.bbf32bbb4c5c76878149.prod.konflux-ocp-ci.dev" "10.133.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.144:59138 10.132.0.144:443 10.132.0.10:57826 - v1-completions-path [2026-11-03T04:40:12.939Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 60 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "0ccd51dc-699b-41b1-a258-b00c2f99fcf5" "maas.apps.bbf32bbb4c5c76878149.prod.konflux-ocp-ci.dev" "10.133.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.144:58054 10.132.0.144:443 10.132.0.10:57834 - v1-completions-path 2026-11-03T04:40:27.711222Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:40:44.619684Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-11-03T04:40:57.726Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 65 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "d324d86f-6848-4454-a3bd-58049ba7b146" "maas.apps.bbf32bbb4c5c76878149.prod.konflux-ocp-ci.dev" "10.133.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.144:58054 10.132.0.144:443 10.132.0.10:54670 - v1-completions-path 2026-11-03T04:41:16.041046Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:41:27.055356Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:41:31.834994Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:42:01.951947Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:42:06.836608Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:42:21.079481Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:42:30.432436Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:42:38.762842Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:42:48.336993Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:42:58.419902Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:43:15.731022Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:43:22.166028Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:43:36.354310Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-11-03T04:43:36.345Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 40 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "ceceffd0-7b65-433a-8f42-8131cc636722" "maas.apps.bbf32bbb4c5c76878149.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.144:443 10.132.0.10:46076 - v1-completions-path [2026-11-03T04:43:45.492Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 76 0 35 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "6d1370ea-c36d-4c4d-8c0d-b9942bdc52c2" "maas.apps.bbf32bbb4c5c76878149.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.144:443 10.132.0.10:47794 - v1-completions-path 2026-11-03T04:43:53.885143Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:44:17.140906Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:44:28.459020Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:44:31.569669Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T04:44:31.569707Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T04:44:31.569723Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T04:44:31.569728Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T04:44:31.455Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 114 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "48f25bd6-64d0-404f-b212-89f980fde30d" "maas.apps.bbf32bbb4c5c76878149.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.144:60288 10.132.0.144:443 10.132.0.10:58932 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T04:44:47.302326Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:44:55.550051Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-11-03T04:47:17.239Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 51 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "3c36c6d7-9e80-4a10-b7ca-05a8531039cc" "maas.apps.bbf32bbb4c5c76878149.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.144:443 10.132.0.10:41730 - v1-completions-path 2026-11-03T04:48:28.649763Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T04:48:28.649789Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T04:48:28.649792Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T04:48:28.649795Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-11-03T04:48:28.604Z] "POST /llm/e2e-trlp-test-simulated/v1/completions HTTP/1.1" 403 - - "-" 73 113 45 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "deadde1c-580d-4294-9655-d4fa8f1075c6" "maas.apps.bbf32bbb4c5c76878149.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-trlp-test-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.144:443 10.132.0.10:50450 - v1-completions-path 2026-11-03T04:48:49.670103Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:49:07.049755Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:49:11.752087Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-11-03T04:49:14.478Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 460 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "fc5633a4-b3f8-41d9-a645-2f344154bfbe" "maas.apps.bbf32bbb4c5c76878149.prod.konflux-ocp-ci.dev" "10.133.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.144:45696 10.132.0.144:443 10.132.0.10:56750 - v1-completions-path 2026-11-03T04:49:25.514252Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:49:42.080511Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:49:47.193868Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:50:07.460249Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:50:18.216714Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:50:24.715098Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:50:37.116518Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T04:50:37.116554Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T04:50:37.116570Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T04:50:37.116575Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T04:50:36.996Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 67 309 119 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "681d1f18-31d2-4951-b740-5f13bbc4cebe" "maas.apps.bbf32bbb4c5c76878149.prod.konflux-ocp-ci.dev" "10.134.0.243:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.144:45666 10.132.0.144:443 10.132.0.10:47708 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T04:50:53.912980Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:51:00.464673Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:51:06.919021Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:51:15.268621Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T04:51:15.268657Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T04:51:15.268675Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T04:51:15.268680Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T04:51:15.154Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 68 310 114 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "c6306439-0359-466e-964d-51e49579ebbe" "maas.apps.bbf32bbb4c5c76878149.prod.konflux-ocp-ci.dev" "10.134.0.243:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.144:45666 10.132.0.144:443 10.132.0.10:36968 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T04:51:21.037Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 62 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "bfede5b6-6afa-4ba0-a54b-4bf1f1dd5f6c" "maas.apps.bbf32bbb4c5c76878149.prod.konflux-ocp-ci.dev" "10.133.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.144:59138 10.132.0.144:443 10.132.0.10:36990 - v1-completions-path 2026-11-03T04:51:39.036641Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:51:49.706943Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:51:56.667257Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:52:03.122889Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:52:11.095131Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T04:52:11.095170Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T04:52:11.095185Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T04:52:11.095190Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T04:52:10.939Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 313 155 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "e7f765f0-f0c5-4387-b41a-5fd99f07fc2f" "maas.apps.bbf32bbb4c5c76878149.prod.konflux-ocp-ci.dev" "10.134.0.243:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.144:58536 10.132.0.144:443 10.132.0.10:53792 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T04:52:14.968634Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:52:38.370731Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:52:49.247357Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:52:52.613894Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:53:12.359075Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:53:18.789625Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:53:42.621256Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:54:02.803962Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:54:13.442165Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:54:26.095974Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:54:45.992092Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:54:56.694879Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:55:07.476098Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:55:18.360522Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:55:28.947499Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-03T04:55:39.740511Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-11-03T04:55:51.957Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 44 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "7b83d30b-23e7-4607-869d-528ed3ea2e0f" "maas.apps.bbf32bbb4c5c76878149.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.144:443 10.132.0.10:59030 - v1-completions-path