2026-10-30T06:26:29.261048Z info FLAG: --concurrency="0" 2026-10-30T06:26:29.261066Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T06:26:29.261070Z info FLAG: --help="false" 2026-10-30T06:26:29.261072Z info FLAG: --log_as_json="false" 2026-10-30T06:26:29.261074Z info FLAG: --log_caller="" 2026-10-30T06:26:29.261076Z info FLAG: --log_output_level="default:info" 2026-10-30T06:26:29.261078Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T06:26:29.261083Z info FLAG: --log_target="[stdout]" 2026-10-30T06:26:29.261086Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T06:26:29.261087Z info FLAG: --outlierLogPath="" 2026-10-30T06:26:29.261089Z info FLAG: --profiling="true" 2026-10-30T06:26:29.261092Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T06:26:29.261093Z info FLAG: --proxyLogLevel="warning" 2026-10-30T06:26:29.261095Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T06:26:29.261097Z info FLAG: --stsPort="0" 2026-10-30T06:26:29.261099Z info FLAG: --templateFile="" 2026-10-30T06:26:29.261101Z info FLAG: --tokenManagerPlugin="" 2026-10-30T06:26:29.261104Z info FLAG: --vklog="0" 2026-10-30T06:26:29.261106Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T06:26:29.261246Z info Proxy role ips=[10.134.1.190] type=router id=maas-default-gateway-openshift-default-59f957fdd7-vfwql.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T06:26:29.261293Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T06:26:29.262356Z info cpu limit detected as 2, setting concurrency 2026-10-30T06:26:29.262579Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T06:26:29.262590Z info JWT policy is third-party-jwt 2026-10-30T06:26:29.262592Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T06:26:29.264116Z info Starting default Istio SDS Server 2026-10-30T06:26:29.264144Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T06:26:29.264161Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T06:26:29.264579Z info Opening status port 15020 2026-10-30T06:26:29.265354Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T06:26:29.266900Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T06:26:29.267780Z info Starting proxy agent 2026-10-30T06:26:29.267815Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T06:26:29.277337Z info sds Starting SDS grpc server 2026-10-30T06:26:29.277370Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T06:26:29.331914Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T06:26:29.573983Z info ads ADS: new connection for node:1 2026-10-30T06:26:29.574945Z info ads ADS: new connection for node:2 2026-10-30T06:26:29.609171Z info cache generated new workload certificate resourceName=default latency=343.910148ms ttl=23h59m59.390832022s 2026-10-30T06:26:29.609201Z info cache Root cert has changed, start rotating root cert 2026-10-30T06:26:29.609226Z info cache returned workload certificate from cache ttl=23h59m59.390777222s 2026-10-30T06:26:29.609233Z info cache returned workload trust anchor from cache ttl=23h59m59.390767437s 2026-10-30T06:26:29.609330Z info cache returned workload trust anchor from cache ttl=23h59m59.390670812s 2026-10-30T06:26:29.609671Z info cache returned workload trust anchor from cache ttl=23h59m59.390329951s 2026-10-30T06:26:29.721757Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:26:35.773700Z error failed scraping envoy metrics: error scraping http://localhost:15090/stats/prometheus: Get "http://localhost:15090/stats/prometheus": dial tcp [::1]:15090: connect: connection refused 2026-10-30T06:26:38.252164Z info Readiness succeeded in 8.993794557s 2026-10-30T06:26:38.252452Z info Envoy proxy is ready 2026-10-30T06:26:41.246960Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:27:55.023227Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:35:18.360511Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:35:41.691809Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:36:13.040453Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:36:28.656362Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T06:36:28.656389Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T06:36:28.656393Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T06:36:28.656397Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-10-30T06:36:28.580Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 403 - - "-" 61 103 75 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "97a2fa2b-0626-4fb6-b36c-ea52c783a79d" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.190:443 10.132.0.20:45782 - v1-catch-all-path 2026-10-30T06:36:29.218243Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:36:31.789057Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:36:39.565683Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:36:39.565723Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:36:39.565737Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T06:36:39.565743Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:36:39.664044Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-10-30T06:36:39.474Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 324 90 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "c2f2ba0f-66fb-4f67-b7d6-6414443177d5" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.132.0.48:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.190:41106 10.134.1.190:443 10.132.0.20:48940 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T06:36:39.588Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 75 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "5058b95f-0cba-46f9-a127-7201ebf014cb" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.1.190:34890 10.134.1.190:443 10.132.0.20:48956 - v1-catch-all-path 2026-10-30T06:36:42.525272Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:36:47.906873Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-10-30T06:36:47.848Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 58 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "19191c14-9110-4312-a225-0b62341f1937" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.1.190:34904 10.134.1.190:443 10.132.0.20:48726 - v1-catch-all-path 2026-10-30T06:37:00.504211Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:37:02.957822Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:37:17.862648Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:37:28.794315Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:37:31.379040Z info ads ADS: new connection for node:3 2026-10-30T06:37:31.379240Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T06:37:31.379271Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T06:37:35.344888Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:38:13.207707Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:38:14.066070Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:38:14.066120Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:38:14.066142Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:38:14.066150Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T06:38:13.940Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 313 126 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "e11a9dd3-b739-48ba-b4e9-510c25f745cb" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.132.0.48:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.190:44024 10.134.1.190:443 10.132.0.20:53516 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T06:38:17.619Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 334 82 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "5aa97369-abca-49e3-96b8-abe8ca3a5263" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.133.0.238:8000" outbound|54321||e2e-prio-pool-da0a2d06-inference-pool-ip-964c2513.llm.svc.cluster.local 10.134.1.190:54064 10.134.1.190:443 10.132.0.20:35302 - v1-model-routing [2026-10-30T06:38:18.509Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 341 62 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "281c561a-4621-4e48-9db3-3e0131fedf85" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.133.0.238:8000" outbound|54321||e2e-prio-pool-da0a2d06-inference-pool-ip-964c2513.llm.svc.cluster.local 10.134.1.190:54080 10.134.1.190:443 10.132.0.20:35316 - v1-model-routing 2026-10-30T06:38:32.344068Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:38:32.476057Z info ads ADS: "" 3 terminated 2026-10-30T06:38:34.917045Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-10-30T06:38:35.774591Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T06:38:34.864Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 66 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "ee33f46e-b4aa-4a78-bf65-0dd5d2816c31" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.133.0.42:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.1.190:51794 10.134.1.190:443 10.132.0.20:51490 - v1-completions-path 2026-10-30T06:38:43.718649Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:40:00.754119Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:40:00.754158Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:40:00.754177Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T06:40:00.754182Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T06:40:00.678Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 33 326 75 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "7d38d981-b043-4737-b270-7add0ecb0c33" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.132.0.48:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.190:44808 10.134.1.190:443 10.132.0.20:47920 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T06:40:05.607362Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:40:23.735391Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:40:38.609717Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:40:50.672794Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:40:54.334878Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:40:54.334920Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:40:54.334938Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T06:40:54.334944Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T06:40:54.278Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 56 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "baf72c38-ddee-4501-be17-e2132f43666a" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.132.0.48:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.190:44808 10.134.1.190:443 10.132.0.20:38260 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T06:40:54.752219Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:41:06.448410Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:41:10.318316Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:41:18.191298Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:41:18.191345Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:41:18.191369Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:41:18.191376Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T06:41:18.140Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 524 50 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "04af1031-ca86-43a6-9989-0ce3605a77e1" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.132.0.48:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.190:44808 10.134.1.190:443 10.132.0.20:57124 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T06:41:30.228832Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:41:32.913846Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:41:32.913882Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:41:32.913896Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:41:32.913902Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T06:41:33.595467Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-10-30T06:41:32.839Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 323 74 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "5f986301-4367-437d-bd7e-2be812d5df42" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.132.0.48:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.190:50480 10.134.1.190:443 10.132.0.20:55302 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T06:41:33.539Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 290 63 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "9cfaab3f-d2d4-4abc-ae0e-33476f96fd94" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.133.0.42:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.1.190:37176 10.134.1.190:443 10.132.0.20:55316 - v1-completions-path [2026-10-30T06:42:15.701Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 66 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "6d9822bd-d543-49d2-9bb4-0b6bd01137b6" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.133.0.42:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.1.190:37176 10.134.1.190:443 10.132.0.20:36262 - v1-completions-path 2026-10-30T06:42:17.067686Z info ads ADS: new connection for node:4 2026-10-30T06:42:17.071566Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-10-30T06:42:17.071614Z info cache read certificate from file resource=file-root:system 2026-10-30T06:42:18.036513Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:42:19.773094Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:42:20.605497Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"285","StreamId":"10962846381841021735"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=24 [2026-10-30T06:42:20.591Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 13 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "100bcd90-5e42-4efe-a920-09b87b856de6" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.134.1.190:443 10.132.0.20:36274 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-10-30T06:43:28.742235Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:43:28.742269Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:43:28.742283Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T06:43:28.742288Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:43:28.742452Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T06:43:28.639Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1732 103 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "b5efe144-989c-4fdc-9837-ffdd5e2fdeb2" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.1.190:40410 10.134.1.190:443 10.132.0.20:58930 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T06:43:48.424287Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:43:48.521811Z info ads ADS: "" 4 terminated 2026-10-30T06:43:50.119808Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:43:57.042486Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:44:14.076934Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:44:18.938098Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T06:44:24.907Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 302 109 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "c48ec725-22b3-443c-a4e6-a9353467e248" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.190:45790 10.134.1.190:443 10.132.0.20:54832 - v1-completions-path 2026-10-30T06:44:39.076050Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:44:53.105509Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T06:44:58.347Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 60 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "cf75771f-9d01-4bbc-b572-ff07221f79ce" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.190:45790 10.134.1.190:443 10.132.0.20:49382 - v1-completions-path [2026-10-30T06:44:59.667Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 56 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "f14ef6e4-3e46-4798-85c5-893666802436" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.190:38706 10.134.1.190:443 10.132.0.20:49394 - v1-completions-path 2026-10-30T06:45:16.424374Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:45:26.799169Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:45:31.657055Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:45:34.872038Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:45:34.872076Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:45:34.872091Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:45:34.872096Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T06:45:34.795Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 317 77 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "4506657e-4438-42f1-a8e1-e9ec1c67a4ba" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.132.0.48:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.190:51224 10.134.1.190:443 10.132.0.20:58558 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T06:45:35.139Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 56 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "9ee56912-1f8a-451c-bb61-1f2feb9f1ab0" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.190:38706 10.134.1.190:443 10.132.0.20:58572 - v1-completions-path 2026-10-30T06:45:48.119688Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:45:50.545798Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:46:09.675161Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:46:14.624889Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:46:30.548674Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T06:46:45.242Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 288 58 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "9881e095-3701-4709-a592-37c4f0145824" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.133.0.42:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.1.190:51794 10.134.1.190:443 10.132.0.20:40682 - v1-completions-path 2026-10-30T06:46:46.590395Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:46:58.153366Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:47:01.636596Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:47:22.574689Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:47:25.566892Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:47:30.895931Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:47:30.895977Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:47:30.896002Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T06:47:30.896009Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T06:47:30.825Z] "DELETE /v1/api-keys/800dac9b-baad-47f3-a06b-c885c98c9fb8 HTTP/1.1" 200 - via_upstream - "-" 0 375 70 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "0269235e-998f-41e5-b3d9-92e27b5067e9" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.132.0.48:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.190:41454 10.134.1.190:443 10.132.0.20:59796 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T06:47:37.563535Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:47:55.523999Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:48:05.821283Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:48:25.959964Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:48:29.343909Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:49:59.676018Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:50:03.376244Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:51:33.184240Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:51:47.857187Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:51:50.328826Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:52:09.223861Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:52:21.381106Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:52:24.419783Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:52:24.524911Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:52:24.524964Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:52:24.524990Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T06:52:24.524997Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T06:52:24.404Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 72 315 120 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "69607bc0-b440-414c-be05-5bd03be8535b" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.132.0.48:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.190:45290 10.134.1.190:443 10.132.0.20:45858 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T06:52:27.337Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 59 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "1df7846f-363e-4a53-b843-a0384725f74a" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.190:45790 10.134.1.190:443 10.132.0.20:39568 - v1-completions-path 2026-10-30T06:52:28.022110Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:52:28.022147Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:52:28.022164Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:52:28.022169Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T06:52:27.950Z] "DELETE /v1/api-keys/994a3b8e-1340-4cb0-bcad-340f3e7ab8c0 HTTP/1.1" 200 - via_upstream - "-" 0 369 71 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "c8d0a869-7897-4cd4-845a-7914102b5d8a" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.132.0.169:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.190:53766 10.134.1.190:443 10.132.0.20:39580 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T06:52:40.059006Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:52:50.781236Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:52:55.494455Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:52:59.607656Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:52:59.607704Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:52:59.607726Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T06:52:59.607733Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T06:52:59.533Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 67 310 74 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "708f676d-a0ba-4072-9349-65f3cf4e64b5" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.132.0.169:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.190:39588 10.134.1.190:443 10.132.0.20:39632 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T06:53:00.823Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 57 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "e054c925-0d5b-4db4-9ff3-d0aaede80134" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.190:38706 10.134.1.190:443 10.132.0.20:39646 - v1-completions-path 2026-10-30T06:53:12.977120Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:53:17.767117Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:53:21.394182Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T06:53:27.468Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 61 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "37704781-c20a-42e7-adca-898b7d1cdef7" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.190:45790 10.134.1.190:443 10.132.0.20:52710 - v1-completions-path [2026-10-30T06:53:28.095Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 54 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "a3c382e3-1354-46f6-8bc4-94be915959fe" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.190:38706 10.134.1.190:443 10.132.0.20:52714 - v1-completions-path [2026-10-30T06:53:28.988Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 57 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "a234ae79-2f06-4a6d-83ea-5b133f8c5494" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.190:38706 10.134.1.190:443 10.132.0.20:52726 - v1-completions-path [2026-10-30T06:53:30.824Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 60 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "d98e0cd7-9a28-49e9-9d9a-6d0af52bca10" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.190:45790 10.134.1.190:443 10.132.0.20:52740 - v1-completions-path [2026-10-30T06:53:31.005Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 63 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "ed186773-5128-426c-9752-5a2773faaf8d" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.190:45790 10.134.1.190:443 10.132.0.20:52752 - v1-completions-path [2026-10-30T06:53:31.191Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 60 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "91fcca84-9bc5-410f-be0a-ea6ce132e288" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.190:45790 10.134.1.190:443 10.132.0.20:52766 - v1-completions-path 2026-10-30T06:53:43.871315Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:53:54.111464Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:53:58.866494Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:54:02.543694Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:54:07.310546Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T06:54:12.569Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 66 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "96bf55cf-75dc-4f58-9bab-df5abeedce42" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.190:45790 10.134.1.190:443 10.132.0.20:51820 - v1-completions-path [2026-10-30T06:54:12.760Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 62 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "9a32c1f5-27c4-4602-a544-435175cbb6a9" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.190:38706 10.134.1.190:443 10.132.0.20:51834 - v1-completions-path [2026-10-30T06:54:12.943Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 62 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "0b902c6e-ce7c-4a3b-ba02-d3578c22330f" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.190:38706 10.134.1.190:443 10.132.0.20:51848 - v1-completions-path 2026-10-30T06:54:13.734013Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:54:13.734048Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:54:13.734064Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T06:54:13.734069Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T06:54:13.644Z] "DELETE /v1/api-keys/27a6291c-5518-41fe-8bb7-f17567fedeb6 HTTP/1.1" 200 - via_upstream - "-" 0 367 89 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "6010278f-5031-4250-ad04-c9b3cddec1e4" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "10.133.0.255:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.190:37020 10.134.1.190:443 10.132.0.20:51864 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T06:54:25.712632Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:54:36.006797Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:54:38.398407Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:54:48.266666Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:54:52.709471Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:55:08.642980Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:55:11.189410Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:55:29.430183Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:55:39.673156Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:55:50.110515Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:56:00.345438Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:56:03.613625Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-10-30T06:56:10.757386Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:56:20.990230Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:56:31.408251Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:56:41.723795Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:56:52.065501Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:57:02.546270Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:57:07.489292Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T06:57:07.489320Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T06:57:07.489323Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T06:57:07.489327Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-10-30T06:57:07.465Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 401 - - "-" 66 88 24 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "0361c203-13a0-4c2a-b7ed-a1ac87633409" "maas.apps.32c8459a3857cfd60680.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.190:443 10.132.0.20:42332 - v1-completions-path