2026-10-31T19:48:37.308043Z info FLAG: --concurrency="0" 2026-10-31T19:48:37.308062Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-31T19:48:37.308066Z info FLAG: --help="false" 2026-10-31T19:48:37.308068Z info FLAG: --log_as_json="false" 2026-10-31T19:48:37.308070Z info FLAG: --log_caller="" 2026-10-31T19:48:37.308072Z info FLAG: --log_output_level="default:info" 2026-10-31T19:48:37.308074Z info FLAG: --log_stacktrace_level="default:none" 2026-10-31T19:48:37.308078Z info FLAG: --log_target="[stdout]" 2026-10-31T19:48:37.308080Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-31T19:48:37.308082Z info FLAG: --outlierLogPath="" 2026-10-31T19:48:37.308084Z info FLAG: --profiling="true" 2026-10-31T19:48:37.308086Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-31T19:48:37.308088Z info FLAG: --proxyLogLevel="warning" 2026-10-31T19:48:37.308090Z info FLAG: --serviceCluster="istio-proxy" 2026-10-31T19:48:37.308092Z info FLAG: --stsPort="0" 2026-10-31T19:48:37.308094Z info FLAG: --templateFile="" 2026-10-31T19:48:37.308095Z info FLAG: --tokenManagerPlugin="" 2026-10-31T19:48:37.308098Z info FLAG: --vklog="0" 2026-10-31T19:48:37.308100Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-31T19:48:37.308268Z info Proxy role ips=[10.132.0.250] type=router id=maas-default-gateway-openshift-default-7f694df8bc-ztb8k.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-31T19:48:37.308319Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-31T19:48:37.309456Z info cpu limit detected as 2, setting concurrency 2026-10-31T19:48:37.309663Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-31T19:48:37.309675Z info JWT policy is third-party-jwt 2026-10-31T19:48:37.309677Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-31T19:48:37.310922Z info Starting default Istio SDS Server 2026-10-31T19:48:37.310963Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-31T19:48:37.310987Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-31T19:48:37.311922Z info Opening status port 15020 2026-10-31T19:48:37.312383Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-31T19:48:37.313741Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-31T19:48:37.314829Z info Starting proxy agent 2026-10-31T19:48:37.314862Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-31T19:48:37.324298Z info sds Starting SDS grpc server 2026-10-31T19:48:37.324316Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-31T19:48:37.374534Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-31T19:48:37.441314Z info cache generated new workload certificate resourceName=default latency=129.095194ms ttl=23h59m59.558688821s 2026-10-31T19:48:37.441353Z info cache Root cert has changed, start rotating root cert 2026-10-31T19:48:37.441390Z info cache returned workload trust anchor from cache ttl=23h59m59.558610679s 2026-10-31T19:48:37.518342Z info ads ADS: new connection for node:1 2026-10-31T19:48:37.518448Z info cache returned workload certificate from cache ttl=23h59m59.481552795s 2026-10-31T19:48:37.518738Z info ads ADS: new connection for node:2 2026-10-31T19:48:37.518808Z info cache returned workload trust anchor from cache ttl=23h59m59.481193365s 2026-10-31T19:48:37.595136Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T19:48:45.163258Z info Readiness succeeded in 7.858041412s 2026-10-31T19:48:45.163565Z info Envoy proxy is ready 2026-10-31T19:48:48.454583Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T19:49:25.785669Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T19:49:30.241Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 144 409 84 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "29bb06e4-ea7c-4217-80d3-ef9274569f59" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "10.134.0.50:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.132.0.250:38674 10.132.0.250:443 10.132.0.17:33676 - v1-chat-completions-path 2026-10-31T19:49:44.238725Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T19:49:48.831513Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T19:49:50.743110Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T19:49:50.743145Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T19:49:50.743164Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T19:49:50.743169Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T19:49:51.035408Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-10-31T19:49:50.599Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 323 143 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "db7cb1a4-e982-4c6e-9c4c-125bcb85dfbb" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "10.132.0.35:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.250:59312 10.132.0.250:443 10.132.0.17:41262 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T19:49:50.947Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 87 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "5f1d5f01-66a7-4675-8f16-2844ef41d4c5" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "10.134.0.48:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.250:58978 10.132.0.250:443 10.132.0.17:41266 - v1-catch-all-path 2026-10-31T19:49:58.235471Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T19:50:00.245318Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 2026-10-31T19:50:01.031417Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-10-31T19:50:01.035303Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-10-31T19:50:00.152Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 92 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "a09e3e19-bb55-43c2-83c5-317381aaae03" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "10.134.0.48:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.250:39122 10.132.0.250:443 10.132.0.17:42696 - v1-catch-all-path [2026-10-31T19:50:00.964Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 70 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "fe55f03f-ff17-4332-9c83-4a9359ea9b5e" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "10.134.0.48:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.250:39122 10.132.0.250:443 10.132.0.17:42708 - v1-catch-all-path 2026-10-31T19:50:02.217267Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-10-31T19:50:02.112Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 104 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "a9b2a43c-0930-4b19-82e7-b337c8e42c88" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "10.134.0.48:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.250:39122 10.132.0.250:443 10.132.0.17:42720 - v1-catch-all-path 2026-10-31T19:50:11.639553Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T19:50:20.906209Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T19:50:37.577257Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T19:50:49.311610Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T19:50:52.637812Z info ads ADS: new connection for node:3 2026-10-31T19:50:52.637977Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-31T19:50:52.638005Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-31T19:50:56.197911Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T19:51:35.221282Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T19:51:59.316812Z info ads ADS: "" 3 terminated 2026-10-31T19:51:59.742549Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T19:52:04.584670Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T19:52:18.117282Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T19:55:17.501076Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T19:55:45.570830Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T19:56:10.373754Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T19:56:26.038385Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T19:56:37.356234Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T19:56:37.356272Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T19:56:37.356290Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T19:56:37.356297Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-31T19:56:37.243Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 84 327 113 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "68781879-059b-4bf3-bd39-9ba2ffb9987a" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "10.132.0.35:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.250:43684 10.132.0.250:443 10.132.0.17:52278 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T19:56:39.391074Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T19:56:51.344680Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T19:56:56.259547Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T19:57:00.712Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 60 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "a9b25730-ef88-4d23-847c-3c428ae0f050" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.250:443 10.132.0.17:34200 - v1-completions-path 2026-10-31T19:57:14.556603Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T19:58:01.220Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 288 82 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "250758a2-3ce7-4ab4-921d-c547018196bc" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "10.133.0.36:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.250:37396 10.132.0.250:443 10.132.0.17:57258 - v1-completions-path 2026-10-31T19:58:03.003032Z info ads ADS: new connection for node:4 2026-10-31T19:58:03.007087Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-10-31T19:58:03.007132Z info cache read certificate from file resource=file-root:system 2026-10-31T19:58:06.960639Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T19:58:08.018686Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T19:58:27.491385Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T19:58:27.491414Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T19:58:27.491423Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T19:58:27.491427Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T19:58:27.491577Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-10-31T19:58:27.364Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1718 127 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "a500b7cb-14c9-43ed-9c09-2cb3918f8a61" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.250:32998 10.132.0.250:443 10.132.0.17:52944 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-31T19:59:03.109444Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T19:59:03.109474Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T19:59:03.109484Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T19:59:03.109488Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T19:59:03.109634Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-10-31T19:59:02.995Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1716 114 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "45a8ad9e-b701-402f-ab94-e4ade3a77e04" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.250:32998 10.132.0.250:443 10.132.0.17:37014 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-31T19:59:19.346108Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-10-31T19:59:19.346907Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T19:59:19.346932Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T19:59:19.346942Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T19:59:19.346946Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T19:59:19.347091Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-31T19:59:19.225Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1716 121 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "e27bb65b-a4be-4897-9a99-871889c27628" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.250:41478 10.132.0.250:443 10.132.0.17:57438 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-31T19:59:57.763362Z info ads ADS: "" 4 terminated with stream closed 2026-10-31T19:59:57.815555Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:00:00.344177Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:00:11.019456Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T20:00:15.572Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 70 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "a55f67e4-be06-4272-aecc-ba3a3120165d" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "10.134.0.49:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.250:41370 10.132.0.250:443 10.132.0.17:45880 - v1-completions-path 2026-10-31T20:00:29.556290Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:00:34.153624Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T20:00:39.497Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 298 66 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "88b546c8-ffdb-42ab-ab56-cdf455ac9a42" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "10.134.0.49:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.250:55986 10.132.0.250:443 10.132.0.17:42754 - v1-completions-path 2026-10-31T20:00:55.070101Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:01:11.978039Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T20:01:20.521Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 62 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "62c89dbc-e58f-4e74-9fae-11a25c383657" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "10.134.0.49:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.250:55986 10.132.0.250:443 10.132.0.17:56726 - v1-completions-path [2026-10-31T20:01:23.226Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 59 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "ebfe1dc7-fc67-4a12-8167-8e79ccbde836" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "10.134.0.49:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.250:55986 10.132.0.250:443 10.132.0.17:56734 - v1-completions-path [2026-10-31T20:01:24.252Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 56 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "b28def6a-8495-4de0-8b2f-ab89c29b290a" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "10.134.0.49:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.250:41370 10.132.0.250:443 10.132.0.17:55514 - v1-completions-path 2026-10-31T20:01:42.895499Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:01:53.497810Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:01:59.808810Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:02:10.695844Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T20:02:10.695879Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T20:02:10.695892Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T20:02:10.695897Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-31T20:02:10.573Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 317 122 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "c270ab54-f9cd-4ed8-83e6-c051fb381337" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "10.132.0.35:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.250:35486 10.132.0.250:443 10.132.0.17:46392 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T20:02:28.472796Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:02:32.987501Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:02:46.363348Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:02:52.835212Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:03:07.679821Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:03:12.137263Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:03:24.180193Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:03:38.871912Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T20:03:44.257Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 66 18 66 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "0b70f380-a87f-4946-9dad-66c07aa1222a" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.250:443 10.132.0.17:52272 - v1-completions-path 2026-10-31T20:03:45.300064Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:04:03.560583Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:04:08.911893Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T20:04:12.743Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 76 0 37 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "82ad0e44-14d8-4f62-b3d1-c1d570754676" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.250:443 10.132.0.17:45488 - v1-completions-path 2026-10-31T20:04:20.185519Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:04:46.928326Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:04:57.608650Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:05:16.782546Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:05:22.962701Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:07:49.150544Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:09:16.618555Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:09:34.009666Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:09:39.029979Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:09:39.312317Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T20:09:39.312370Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T20:09:39.312397Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T20:09:39.312405Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-31T20:09:39.149Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 65 307 163 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "322bae5d-5a50-41d5-826c-115b560de4e1" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "10.132.0.35:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.250:49426 10.132.0.250:443 10.132.0.17:43016 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T20:09:50.784775Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:10:08.170279Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:10:13.395206Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:10:19.706137Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T20:10:19.706176Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T20:10:19.706209Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T20:10:19.706215Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-31T20:10:19.590Z] "DELETE /v1/api-keys/0f4bf7e3-b9b7-4a35-9adc-1bb4ce0d7c1f HTTP/1.1" 200 - via_upstream - "-" 0 369 115 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "3efdb4f0-d3f4-4aee-bde0-496ebfb077dd" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "10.133.1.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.250:60718 10.132.0.250:443 10.132.0.17:54212 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T20:10:33.403312Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:10:44.235459Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:10:50.517928Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:11:20.782011Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:11:27.115442Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:11:33.549456Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T20:11:42.218Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 60 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "4d7a395c-67dc-4ae3-bd31-aa35d6caaee3" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "10.134.0.49:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.250:55986 10.132.0.250:443 10.132.0.17:46262 - v1-completions-path [2026-10-31T20:11:44.200Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 55 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "8cad6efc-d28d-4022-bf2e-cde272af45b9" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.250:443 10.132.0.17:38310 - v1-completions-path [2026-10-31T20:11:47.366Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 62 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "be834b75-d29c-49f7-b14b-b16c7015e176" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "10.134.0.49:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.250:55986 10.132.0.250:443 10.132.0.17:38316 - v1-completions-path [2026-10-31T20:11:47.723Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 66 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "11208ea3-e918-4181-b45c-da9f9037bf74" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "10.134.0.49:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.250:41370 10.132.0.250:443 10.132.0.17:38318 - v1-completions-path 2026-10-31T20:11:52.121162Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T20:11:52.121232Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T20:11:52.121250Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T20:11:52.121255Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-31T20:11:51.965Z] "DELETE /v1/api-keys/baf80c11-8311-4b00-b8cc-37bb08b133d8 HTTP/1.1" 200 - via_upstream - "-" 0 365 155 - "34.228.250.194,10.132.0.17" "python-requests/2.34.2" "42a46b3f-9e35-4df6-8a3f-9ebf2707887e" "maas.apps.4cc86155c781c54889b2.prod.konflux-ocp-ci.dev" "10.134.0.255:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.250:48934 10.132.0.250:443 10.132.0.17:38336 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T20:12:05.614016Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:12:16.323395Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:12:23.088541Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:12:29.167176Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:12:41.137129Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:13:04.734667Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:13:15.359716Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:13:20.196294Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:13:38.511338Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:13:45.049039Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:14:08.659878Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:14:30.901126Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:14:50.875668Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:15:01.893798Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:15:13.011135Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:15:23.645538Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:15:34.507787Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:15:45.408135Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:15:56.075417Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T20:16:06.730960Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8