2026-10-31T05:06:41.151251Z info FLAG: --concurrency="0" 2026-10-31T05:06:41.151271Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-31T05:06:41.151275Z info FLAG: --help="false" 2026-10-31T05:06:41.151277Z info FLAG: --log_as_json="false" 2026-10-31T05:06:41.151280Z info FLAG: --log_caller="" 2026-10-31T05:06:41.151282Z info FLAG: --log_output_level="default:info" 2026-10-31T05:06:41.151284Z info FLAG: --log_stacktrace_level="default:none" 2026-10-31T05:06:41.151288Z info FLAG: --log_target="[stdout]" 2026-10-31T05:06:41.151290Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-31T05:06:41.151293Z info FLAG: --outlierLogPath="" 2026-10-31T05:06:41.151295Z info FLAG: --profiling="true" 2026-10-31T05:06:41.151297Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-31T05:06:41.151299Z info FLAG: --proxyLogLevel="warning" 2026-10-31T05:06:41.151301Z info FLAG: --serviceCluster="istio-proxy" 2026-10-31T05:06:41.151303Z info FLAG: --stsPort="0" 2026-10-31T05:06:41.151305Z info FLAG: --templateFile="" 2026-10-31T05:06:41.151307Z info FLAG: --tokenManagerPlugin="" 2026-10-31T05:06:41.151309Z info FLAG: --vklog="0" 2026-10-31T05:06:41.151311Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-31T05:06:41.151452Z info Proxy role ips=[10.133.0.213] type=router id=maas-default-gateway-openshift-default-7d64fcddb7-jvjvc.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-31T05:06:41.151496Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-31T05:06:41.152512Z info cpu limit detected as 2, setting concurrency 2026-10-31T05:06:41.153156Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-31T05:06:41.153170Z info JWT policy is third-party-jwt 2026-10-31T05:06:41.153172Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-31T05:06:41.154782Z info Starting default Istio SDS Server 2026-10-31T05:06:41.154812Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-31T05:06:41.154831Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-31T05:06:41.154950Z info Opening status port 15020 2026-10-31T05:06:41.155808Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-31T05:06:41.156285Z info sds Starting SDS grpc server 2026-10-31T05:06:41.156307Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-31T05:06:41.156980Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-31T05:06:41.158039Z info Starting proxy agent 2026-10-31T05:06:41.158081Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-31T05:06:41.222554Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-31T05:06:41.355674Z info cache generated new workload certificate resourceName=default latency=199.26753ms ttl=23h59m59.644332738s 2026-10-31T05:06:41.355724Z info cache Root cert has changed, start rotating root cert 2026-10-31T05:06:41.355767Z info cache returned workload trust anchor from cache ttl=23h59m59.644233779s 2026-10-31T05:06:41.373015Z info ads ADS: new connection for node:1 2026-10-31T05:06:41.373077Z info cache returned workload certificate from cache ttl=23h59m59.62692469s 2026-10-31T05:06:41.373247Z info ads ADS: new connection for node:2 2026-10-31T05:06:41.373443Z info cache returned workload trust anchor from cache ttl=23h59m59.626559882s 2026-10-31T05:06:41.455260Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:06:49.236147Z info Readiness succeeded in 8.087121758s 2026-10-31T05:06:49.236514Z info Envoy proxy is ready 2026-10-31T05:06:50.369323Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:07:07.283892Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:07:18.911391Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:07:22.473284Z info ads ADS: new connection for node:3 2026-10-31T05:07:22.473452Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-31T05:07:22.473483Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-31T05:07:25.803615Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:08:10.476494Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T05:08:17.581Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 333 102 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "d17e30c4-abb1-4f54-b570-a3ab876df33c" "maas.apps.be049019ee31a6cc9291.prod.konflux-ocp-ci.dev" "10.133.0.214:8000" outbound|54321||e2e-prio-pool-36410873-inference-pool-ip-2851bf73.llm.svc.cluster.local 10.133.0.213:34042 10.133.0.213:443 10.132.0.19:41350 - v1-model-routing [2026-10-31T05:08:33.036Z] "POST /v1/chat/completions HTTP/1.1" 403 - - "-" 131 0 35 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "1d15a293-f8e8-49ab-a303-e3d2157b6abe" "maas.apps.be049019ee31a6cc9291.prod.konflux-ocp-ci.dev" "-" outbound|54321||e2e-prio-pool-36410873-inference-pool-ip-2851bf73.llm.svc.cluster.local - 10.133.0.213:443 10.132.0.19:48224 - v1-model-routing 2026-10-31T05:08:35.320450Z info ads ADS: "" 3 terminated with stream closed 2026-10-31T05:08:35.371913Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:08:41.846181Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:08:54.765806Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:11:13.517531Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T05:11:13.517602Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T05:11:13.517619Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T05:11:13.517624Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-31T05:11:13.440Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 77 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "f1e20439-4d4b-4dff-bbbb-3d7a89f22806" "maas.apps.be049019ee31a6cc9291.prod.konflux-ocp-ci.dev" "10.133.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.213:48352 10.133.0.213:443 10.132.0.19:38116 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T05:11:32.017174Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:11:59.711411Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:12:36.230328Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:12:51.392779Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:13:00.230644Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:13:13.307098Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:13:17.738868Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:13:36.054066Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:16:14.472547Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T05:16:14.472615Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T05:16:14.472634Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T05:16:14.472640Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-31T05:16:14.349Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 88 330 123 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "8e97e2a1-9620-4c45-92b6-66e7b32c55a4" "maas.apps.be049019ee31a6cc9291.prod.konflux-ocp-ci.dev" "10.133.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.213:52402 10.133.0.213:443 10.132.0.19:46762 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T05:17:00.593975Z info ads ADS: new connection for node:4 2026-10-31T05:17:00.597497Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-10-31T05:17:00.597594Z info cache read certificate from file resource=file-root:system 2026-10-31T05:17:04.370039Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:17:05.734414Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:17:25.670473Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T05:17:25.670504Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T05:17:25.670513Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T05:17:25.670517Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T05:17:25.670693Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-10-31T05:17:25.468Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1714 202 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "daa826a0-66a1-432c-8232-9fc2cb769ec5" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.213:45920 10.133.0.213:443 10.132.0.19:38198 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-31T05:17:42.254800Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T05:17:42.254836Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T05:17:42.254851Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T05:17:42.254857Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T05:17:42.255057Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-31T05:17:42.064Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1720 190 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "cadfc8eb-706d-4eda-ba7a-5e8550a50cf5" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.213:38282 10.133.0.213:443 10.132.0.19:50462 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-31T05:18:20.292453Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-10-31T05:18:20.366085Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T05:18:20.366116Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T05:18:20.366126Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T05:18:20.366131Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T05:18:20.366287Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-10-31T05:18:20.167Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1723 198 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "be506ef5-4400-4216-afab-befb6e3f77ae" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.213:45920 10.133.0.213:443 10.132.0.19:35360 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-31T05:18:35.621348Z info ads ADS: "" 4 terminated 2026-10-31T05:18:35.662273Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:18:38.696080Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:18:48.245807Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:19:07.102160Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:19:11.821577Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:19:28.302208Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:19:45.207580Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:19:51.806022Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T05:19:51.806065Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T05:19:51.806084Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T05:19:51.806091Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-31T05:19:51.688Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 324 117 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "2971e27b-5029-46bd-8397-b8d14c203653" "maas.apps.be049019ee31a6cc9291.prod.konflux-ocp-ci.dev" "10.133.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.213:34622 10.133.0.213:443 10.132.0.19:45374 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T05:19:52.800Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 78 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "69c305d8-e2f7-4b1e-9924-e1cd245a7175" "maas.apps.be049019ee31a6cc9291.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.213:54692 10.133.0.213:443 10.132.0.19:45390 - v1-completions-path 2026-10-31T05:20:16.307055Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:20:27.047833Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:20:33.282418Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:21:01.919776Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:21:06.608716Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:21:19.913944Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:21:26.186138Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:21:40.836612Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:21:45.293482Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:21:57.077407Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:22:12.306757Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:22:18.625844Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T05:22:20.903Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 66 18 329 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "274762db-c878-46a8-b24f-dd063291831f" "maas.apps.be049019ee31a6cc9291.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.213:443 10.132.0.19:56148 - v1-completions-path 2026-10-31T05:22:38.629977Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:22:44.129463Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:22:55.232642Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T05:23:05.483Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 298 65 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "2fca603f-11ff-45f6-9e75-44199f822491" "maas.apps.be049019ee31a6cc9291.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.213:42340 10.133.0.213:443 10.132.0.19:38310 - v1-completions-path [2026-10-31T05:23:05.755Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 62 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "40aad092-2316-41ac-a2dc-1a63fe57c549" "maas.apps.be049019ee31a6cc9291.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.213:54692 10.133.0.213:443 10.132.0.19:38312 - v1-completions-path 2026-10-31T05:23:19.542034Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:23:30.287585Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T05:23:45.675Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 37 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "7479cbd6-3eed-4107-9bde-fc36ceedd01d" "maas.apps.be049019ee31a6cc9291.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.213:443 10.132.0.19:51486 - v1-completions-path 2026-10-31T05:23:49.918365Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:23:57.458899Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:26:07.412114Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-10-31T05:26:07.520990Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T05:26:07.521027Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T05:26:07.521048Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T05:26:07.521053Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-31T05:26:07.404Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 80 322 116 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "c6b68dcf-5d76-479a-9996-a46b0173e96e" "maas.apps.be049019ee31a6cc9291.prod.konflux-ocp-ci.dev" "10.133.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.213:36620 10.133.0.213:443 10.132.0.19:48262 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T05:26:22.896655Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:27:52.883313Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:28:12.568625Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:28:17.835825Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:28:31.531429Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:28:45.314785Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:28:50.655330Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T05:28:55.028Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 67 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "77b97a7f-75aa-42b0-95ca-1ff5130e516f" "maas.apps.be049019ee31a6cc9291.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.213:54692 10.133.0.213:443 10.132.0.19:38982 - v1-completions-path 2026-10-31T05:28:56.743355Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T05:28:56.743393Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T05:28:56.743413Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T05:28:56.743420Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-31T05:28:56.628Z] "DELETE /v1/api-keys/05e932d9-8a05-48db-a3b7-d5ca9cf8000b HTTP/1.1" 200 - via_upstream - "-" 0 369 115 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "6af25804-059a-4996-a690-ce512572ad6c" "maas.apps.be049019ee31a6cc9291.prod.konflux-ocp-ci.dev" "10.133.0.233:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.213:40636 10.133.0.213:443 10.132.0.19:38998 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T05:29:10.368653Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:29:21.073950Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:29:27.653277Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T05:29:42.017Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 62 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "c74e317f-8f54-42a5-93ee-6c0ddf3eb1a1" "maas.apps.be049019ee31a6cc9291.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.213:42340 10.133.0.213:443 10.132.0.19:40184 - v1-completions-path 2026-10-31T05:29:57.062400Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:30:03.789485Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:30:10.117044Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:30:18.006975Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T05:30:18.007017Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T05:30:18.007036Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T05:30:18.007041Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-31T05:30:17.888Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 66 308 118 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "e67128e4-6284-4f62-bda3-93b1e441b273" "maas.apps.be049019ee31a6cc9291.prod.konflux-ocp-ci.dev" "10.132.1.8:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.213:42092 10.133.0.213:443 10.132.0.19:37996 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T05:30:19.378Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 82 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "41bca86f-3545-4fad-b5af-a997e53f9f91" "maas.apps.be049019ee31a6cc9291.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.213:54692 10.133.0.213:443 10.132.0.19:48424 - v1-completions-path [2026-10-31T05:30:19.748Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 59 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "46f24c26-861f-47f8-ab09-953a3c9744f3" "maas.apps.be049019ee31a6cc9291.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.213:42340 10.133.0.213:443 10.132.0.19:48428 - v1-completions-path [2026-10-31T05:30:20.106Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 60 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "ee3f1575-fa46-474a-af5a-0ff227ec0715" "maas.apps.be049019ee31a6cc9291.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.213:42340 10.133.0.213:443 10.132.0.19:48438 - v1-completions-path [2026-10-31T05:30:20.749Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 57 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "9c68127d-6b87-47dd-a6c9-77247f43cf45" "maas.apps.be049019ee31a6cc9291.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.213:54692 10.133.0.213:443 10.132.0.19:48444 - v1-completions-path 2026-10-31T05:30:42.018862Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:30:52.535306Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:30:59.470837Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:31:05.840510Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:31:17.933496Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T05:31:26.047Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 62 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "376bd751-c417-4d35-9a03-b38f96af821b" "maas.apps.be049019ee31a6cc9291.prod.konflux-ocp-ci.dev" "10.132.0.57:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.213:54692 10.133.0.213:443 10.132.0.19:38386 - v1-completions-path 2026-10-31T05:31:40.788045Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:31:51.580798Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:31:56.788424Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:32:12.806612Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:32:19.215622Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:32:24.551618Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:32:48.269949Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:32:59.385880Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:33:19.258336Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:33:30.283802Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:33:41.185879Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:33:52.132214Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:34:02.747104Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:34:13.469051Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:34:24.324285Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:34:35.258804Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:34:45.723531Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T05:34:49.914310Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-31T05:34:49.914337Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-31T05:34:49.914340Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-31T05:34:49.914344Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-10-31T05:34:49.891Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 401 - - "-" 66 88 22 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "2a529344-6888-4a12-9ad8-6903da437011" "maas.apps.be049019ee31a6cc9291.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.213:443 10.132.0.19:53580 - v1-completions-path [2026-10-31T05:34:53.491Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 83 - "34.228.250.194,10.132.0.19" "python-requests/2.34.2" "330293a4-7485-42b9-8c3e-b2d66a5d4c21" "maas.apps.be049019ee31a6cc9291.prod.konflux-ocp-ci.dev" "10.133.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.213:49756 10.133.0.213:443 10.132.0.19:53596 - v1-completions-path