2026-11-04T01:56:57.718413Z info FLAG: --concurrency="0" 2026-11-04T01:56:57.718432Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-04T01:56:57.718436Z info FLAG: --help="false" 2026-11-04T01:56:57.718439Z info FLAG: --log_as_json="false" 2026-11-04T01:56:57.718441Z info FLAG: --log_caller="" 2026-11-04T01:56:57.718443Z info FLAG: --log_output_level="default:info" 2026-11-04T01:56:57.718445Z info FLAG: --log_stacktrace_level="default:none" 2026-11-04T01:56:57.718449Z info FLAG: --log_target="[stdout]" 2026-11-04T01:56:57.718451Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-04T01:56:57.718454Z info FLAG: --outlierLogPath="" 2026-11-04T01:56:57.718456Z info FLAG: --profiling="true" 2026-11-04T01:56:57.718458Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-04T01:56:57.718460Z info FLAG: --proxyLogLevel="warning" 2026-11-04T01:56:57.718462Z info FLAG: --serviceCluster="istio-proxy" 2026-11-04T01:56:57.718464Z info FLAG: --stsPort="0" 2026-11-04T01:56:57.718466Z info FLAG: --templateFile="" 2026-11-04T01:56:57.718468Z info FLAG: --tokenManagerPlugin="" 2026-11-04T01:56:57.718471Z info FLAG: --vklog="0" 2026-11-04T01:56:57.718473Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-04T01:56:57.718617Z info Proxy role ips=[10.134.0.250] type=router id=maas-default-gateway-openshift-default-fc8c56547-2gbcz.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-04T01:56:57.718666Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-04T01:56:57.719809Z info cpu limit detected as 2, setting concurrency 2026-11-04T01:56:57.720074Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-04T01:56:57.720086Z info JWT policy is third-party-jwt 2026-11-04T01:56:57.720090Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-04T01:56:57.721641Z info Starting default Istio SDS Server 2026-11-04T01:56:57.721665Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-04T01:56:57.721699Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-04T01:56:57.722569Z info Opening status port 15020 2026-11-04T01:56:57.722646Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-04T01:56:57.723235Z info sds Starting SDS grpc server 2026-11-04T01:56:57.723255Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-04T01:56:57.724429Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-04T01:56:57.725971Z info Starting proxy agent 2026-11-04T01:56:57.726663Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-04T01:56:57.796478Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-04T01:56:57.889319Z info cache generated new workload certificate resourceName=default latency=165.627801ms ttl=23h59m59.110684112s 2026-11-04T01:56:57.889354Z info cache Root cert has changed, start rotating root cert 2026-11-04T01:56:57.889383Z info cache returned workload trust anchor from cache ttl=23h59m59.110617128s 2026-11-04T01:56:57.945973Z info ads ADS: new connection for node:1 2026-11-04T01:56:57.946057Z info cache returned workload certificate from cache ttl=23h59m59.053945071s 2026-11-04T01:56:57.946720Z info ads ADS: new connection for node:2 2026-11-04T01:56:57.946852Z info cache returned workload trust anchor from cache ttl=23h59m59.053149153s 2026-11-04T01:56:58.029536Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:57:06.418179Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:57:07.359996Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-04T01:57:08.230182Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-04T01:57:09.032755Z info Readiness succeeded in 11.317202518s 2026-11-04T01:57:09.033247Z info Envoy proxy is ready 2026-11-04T01:57:13.734015Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-04T01:57:13.734068Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-04T01:57:13.734078Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-04T01:57:13.734083Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-04T01:57:13.612Z] "POST /v1/api-keys HTTP/1.1" 400 - via_upstream - "-" 80 94 121 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "b51912dd-8928-434c-b598-c284a569958f" "maas.apps.aeb4087126d95ff136b4.prod.konflux-ocp-ci.dev" "10.133.0.25:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.250:56582 10.134.0.250:443 10.132.0.18:35590 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T01:57:38.848533Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:57:58.129172Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:58:03.184462Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:58:13.826076Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:58:17.598771Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=26 [2026-11-04T01:58:17.516Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 82 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "61980949-14db-4002-99aa-7b0a0f2ee13e" "maas.apps.aeb4087126d95ff136b4.prod.konflux-ocp-ci.dev" "10.133.0.32:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.250:33730 10.134.0.250:443 10.132.0.18:39318 - v1-catch-all-path 2026-11-04T01:58:18.774158Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=26 2026-11-04T01:58:18.774404Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=26 [2026-11-04T01:58:18.712Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 61 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "77545869-de34-47ed-85b4-0571b732dacf" "maas.apps.aeb4087126d95ff136b4.prod.konflux-ocp-ci.dev" "10.133.0.32:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.250:33730 10.134.0.250:443 10.132.0.18:39342 - v1-catch-all-path 2026-11-04T01:58:27.917714Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:58:37.870610Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:58:55.119622Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:59:06.603069Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:59:09.387122Z info ads ADS: new connection for node:3 2026-11-04T01:59:09.387337Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-04T01:59:09.387371Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-04T01:59:13.783221Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:59:58.431741Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:00:01.961152Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T02:00:01.961186Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T02:00:01.961199Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T02:00:01.961204Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-04T02:00:01.834Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 126 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "e7597c66-a83d-49e5-b2d7-918b79d16c3f" "maas.apps.aeb4087126d95ff136b4.prod.konflux-ocp-ci.dev" "10.133.0.25:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.250:53402 10.134.0.250:443 10.132.0.18:54214 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T02:00:23.486864Z info ads ADS: "" 3 terminated with stream closed 2026-11-04T02:00:23.815659Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:00:28.972883Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:00:46.290606Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:03:27.816671Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:03:58.228973Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:04:28.697793Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:04:42.952775Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:04:58.564288Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:05:11.490092Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:05:16.896295Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-04T02:05:22.380Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 57 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "03e3f498-6af0-4dfd-bfa4-25bcd68d94f9" "maas.apps.aeb4087126d95ff136b4.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.250:443 10.132.0.18:58192 - v1-completions-path 2026-11-04T02:05:36.942559Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-04T02:06:08.477Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 81 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "9de05ea8-68a7-46fb-aa7a-0f44e612187f" "maas.apps.aeb4087126d95ff136b4.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.250:44166 10.134.0.250:443 10.132.0.18:57228 - v1-completions-path 2026-11-04T02:06:24.995216Z info ads ADS: new connection for node:4 2026-11-04T02:06:25.000067Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-04T02:06:25.000106Z info cache read certificate from file resource=file-root:system 2026-11-04T02:06:28.831719Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:06:30.288939Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:06:42.854857Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-04T02:06:42.854891Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-04T02:06:42.854903Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-04T02:06:42.854908Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-04T02:06:42.855106Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-11-04T02:06:42.657Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1533 197 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "e16af100-2dd4-40f2-9d38-04e224751a17" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.250:43790 10.134.0.250:443 10.132.0.18:32830 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-04T02:06:45.277942Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T02:06:45.277971Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T02:06:45.277980Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T02:06:45.277985Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T02:06:45.278176Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-04T02:06:45.082Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1534 196 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "f5253fb5-f218-46ab-966b-b1d189044adc" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.250:43796 10.134.0.250:443 10.132.0.18:32852 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-04T02:07:06.702295Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-04T02:07:06.702326Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-04T02:07:06.702336Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-04T02:07:06.702341Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-04T02:07:06.702490Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-11-04T02:07:06.569Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1534 133 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "c247554f-a9cc-47f8-9ed1-302d99468a5a" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.250:43790 10.134.0.250:443 10.132.0.18:59426 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-04T02:07:35.161685Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-04T02:07:35.229678Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T02:07:35.229708Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T02:07:35.229717Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T02:07:35.229721Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T02:07:35.229887Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-04T02:07:35.100Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1528 129 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "a6e1e987-a016-4a49-a1bd-d11b06c2d82d" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.250:43796 10.134.0.250:443 10.132.0.18:48808 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-04T02:07:56.398387Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T02:07:56.398423Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T02:07:56.398432Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T02:07:56.398437Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T02:07:56.398603Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-04T02:07:56.271Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1532 126 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "a713caa7-558e-4416-a1b8-55d8954c33f0" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.250:43796 10.134.0.250:443 10.132.0.18:56124 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-04T02:08:27.455540Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:08:30.887793Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:08:30.976105Z info ads ADS: "" 4 terminated with stream closed 2026-11-04T02:08:31.206940Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:12:01.699995Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-04T02:12:05.849Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 92 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "5d737088-30c3-4e2b-b6ce-6e6a9b553320" "maas.apps.aeb4087126d95ff136b4.prod.konflux-ocp-ci.dev" "10.133.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.250:35490 10.134.0.250:443 10.132.0.18:57156 - v1-completions-path [2026-11-04T02:12:06.729Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 63 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "0b46ba27-a8bb-4133-a554-fcda00e37378" "maas.apps.aeb4087126d95ff136b4.prod.konflux-ocp-ci.dev" "10.133.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.250:35498 10.134.0.250:443 10.132.0.18:57162 - v1-completions-path 2026-11-04T02:12:21.056239Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:12:26.028629Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:12:50.779393Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:13:03.719817Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-04T02:13:14.760Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 65 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "44857cf5-94ab-47f7-b08c-c25da1de7116" "maas.apps.aeb4087126d95ff136b4.prod.konflux-ocp-ci.dev" "10.133.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.250:35490 10.134.0.250:443 10.132.0.18:47388 - v1-completions-path 2026-11-04T02:13:35.061823Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:13:46.205914Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:13:51.248406Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-04T02:14:03.707Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 85 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "dcb663ab-4d08-4a78-9958-a83ae2345019" "maas.apps.aeb4087126d95ff136b4.prod.konflux-ocp-ci.dev" "10.133.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.250:35490 10.134.0.250:443 10.132.0.18:36482 - v1-completions-path 2026-11-04T02:14:20.929772Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:14:25.877111Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:14:40.019412Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:14:49.644448Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:14:59.520447Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:15:06.327341Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:15:21.709369Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:15:34.415337Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:15:40.905429Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:15:59.507795Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:16:05.336983Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:16:17.228694Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:16:43.065929Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:16:53.992541Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:17:12.709075Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:17:20.858183Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:19:50.460009Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:19:56.819923Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:21:24.006875Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:21:41.376907Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:21:45.808079Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:21:58.454536Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:22:15.609994Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:22:22.037009Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:22:41.586762Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:22:52.349624Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:22:58.519232Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:23:12.141139Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-04T02:23:12.141172Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-04T02:23:12.141185Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-04T02:23:12.141189Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-04T02:23:12.023Z] "DELETE /v1/api-keys/3fddce9d-d8fa-4b22-9f45-a10fec8e7426 HTTP/1.1" 200 - via_upstream - "-" 0 365 117 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "3fe94d38-9f21-4dce-a47d-f92c2c20a135" "maas.apps.aeb4087126d95ff136b4.prod.konflux-ocp-ci.dev" "10.133.0.193:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.250:34982 10.134.0.250:443 10.132.0.18:38242 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T02:23:25.503334Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:23:30.026648Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:23:35.832831Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-04T02:23:42.029Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 64 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "672b3fee-01dd-42ea-b77f-925f8fbe6638" "maas.apps.aeb4087126d95ff136b4.prod.konflux-ocp-ci.dev" "10.133.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.250:35498 10.134.0.250:443 10.132.0.18:48462 - v1-completions-path [2026-11-04T02:23:49.165Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 68 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "de9f4ae7-610d-4c82-9dff-1c0dbe30489a" "maas.apps.aeb4087126d95ff136b4.prod.konflux-ocp-ci.dev" "10.133.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.250:35490 10.134.0.250:443 10.132.0.18:46652 - v1-completions-path [2026-11-04T02:23:49.913Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 66 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "d7aff4f1-b023-42c5-a416-d55f5ffb7e4c" "maas.apps.aeb4087126d95ff136b4.prod.konflux-ocp-ci.dev" "10.133.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.250:35490 10.134.0.250:443 10.132.0.18:46662 - v1-completions-path 2026-11-04T02:24:05.130491Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:24:16.002602Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:24:22.487041Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:24:28.303040Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:24:37.328803Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-11-04T02:24:45.795Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 61 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "c70ce141-e94a-9519-8575-7eec4ab5b422" "maas.apps.aeb4087126d95ff136b4.prod.konflux-ocp-ci.dev" "10.133.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.250:35498 10.134.0.250:443 10.132.0.18:49910 - v1-completions-path 2026-11-04T02:25:00.685603Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:25:11.389412Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:25:16.063713Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:25:34.145429Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:25:40.011309Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:25:58.934045Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:26:20.870465Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:26:42.288181Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:27:02.068166Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:27:12.844360Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:27:23.565620Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:27:34.405130Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:27:45.127446Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:27:55.685217Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:28:02.532391Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-04T02:28:02.532418Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-04T02:28:02.532421Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-04T02:28:02.532425Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-11-04T02:28:02.506Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 401 - - "-" 66 88 26 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "d331f12d-a849-4354-a8e8-b2ce2d0076f2" "maas.apps.aeb4087126d95ff136b4.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.250:443 10.132.0.18:55748 - v1-completions-path 2026-11-04T02:28:12.458816Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2