2026-10-30T04:35:07.777343Z info FLAG: --concurrency="0" 2026-10-30T04:35:07.777359Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T04:35:07.777363Z info FLAG: --help="false" 2026-10-30T04:35:07.777365Z info FLAG: --log_as_json="false" 2026-10-30T04:35:07.777368Z info FLAG: --log_caller="" 2026-10-30T04:35:07.777370Z info FLAG: --log_output_level="default:info" 2026-10-30T04:35:07.777372Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T04:35:07.777376Z info FLAG: --log_target="[stdout]" 2026-10-30T04:35:07.777378Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T04:35:07.777380Z info FLAG: --outlierLogPath="" 2026-10-30T04:35:07.777382Z info FLAG: --profiling="true" 2026-10-30T04:35:07.777384Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T04:35:07.777386Z info FLAG: --proxyLogLevel="warning" 2026-10-30T04:35:07.777389Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T04:35:07.777391Z info FLAG: --stsPort="0" 2026-10-30T04:35:07.777393Z info FLAG: --templateFile="" 2026-10-30T04:35:07.777395Z info FLAG: --tokenManagerPlugin="" 2026-10-30T04:35:07.777397Z info FLAG: --vklog="0" 2026-10-30T04:35:07.777400Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T04:35:07.777525Z info Proxy role ips=[10.134.1.7] type=router id=maas-default-gateway-openshift-default-64d4cc998c-lh7pv.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T04:35:07.777571Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T04:35:07.778541Z info cpu limit detected as 2, setting concurrency 2026-10-30T04:35:07.778833Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T04:35:07.778844Z info JWT policy is third-party-jwt 2026-10-30T04:35:07.778848Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T04:35:07.781036Z info Starting default Istio SDS Server 2026-10-30T04:35:07.781062Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T04:35:07.781094Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T04:35:07.781519Z info Opening status port 15020 2026-10-30T04:35:07.782414Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T04:35:07.783387Z info sds Starting SDS grpc server 2026-10-30T04:35:07.783403Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T04:35:07.783667Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T04:35:07.784444Z info Starting proxy agent 2026-10-30T04:35:07.784471Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T04:35:07.847652Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T04:35:07.984512Z info ads ADS: new connection for node:1 2026-10-30T04:35:07.984951Z info ads ADS: new connection for node:2 2026-10-30T04:35:08.085207Z info cache generated new workload certificate resourceName=default latency=301.687082ms ttl=23h59m59.914797437s 2026-10-30T04:35:08.085245Z info cache Root cert has changed, start rotating root cert 2026-10-30T04:35:08.085292Z info cache returned workload trust anchor from cache ttl=23h59m59.914709046s 2026-10-30T04:35:08.085311Z info cache returned workload certificate from cache ttl=23h59m59.914689188s 2026-10-30T04:35:08.085407Z info cache returned workload trust anchor from cache ttl=23h59m59.914597153s 2026-10-30T04:35:08.085752Z info cache returned workload trust anchor from cache ttl=23h59m59.914249406s 2026-10-30T04:35:08.177616Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:35:16.154428Z info Readiness succeeded in 8.380043084s 2026-10-30T04:35:16.154826Z info Envoy proxy is ready 2026-10-30T04:35:35.833168Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:35:53.281233Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:35:55.741505Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:36:03.953101Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-10-30T04:36:03.865Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 87 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "d1c9756f-8dc3-4d0f-8804-f3d28af6fa62" "maas.apps.d36a3d177388305ff350.prod.konflux-ocp-ci.dev" "10.134.0.30:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.1.7:56778 10.134.1.7:443 10.132.0.20:42896 - v1-catch-all-path 2026-10-30T04:36:06.700773Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:36:12.309544Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-10-30T04:36:12.250Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 59 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "4d2c02d8-75ce-46be-a8ff-9c06e4436b02" "maas.apps.d36a3d177388305ff350.prod.konflux-ocp-ci.dev" "10.134.0.30:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.1.7:56778 10.134.1.7:443 10.132.0.20:36878 - v1-catch-all-path 2026-10-30T04:36:24.745880Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:36:26.972268Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:36:41.750968Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:36:52.536483Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:36:54.551362Z info ads ADS: new connection for node:3 2026-10-30T04:36:54.551725Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T04:36:54.551828Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T04:36:56.224714Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:37:36.550424Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:37:39.473280Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: gRPC status code is not OK thread=24 [2026-10-30T04:37:37.413Z] "POST /maas-api/v1/api-keys HTTP/1.1" 500 - - "-" 70 23 2061 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "d3354c4f-627c-4e08-b904-22c422f0667b" "maas.apps.d36a3d177388305ff350.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.134.1.7:443 10.132.0.20:47538 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T04:38:00.708492Z info ads ADS: "" 3 terminated 2026-10-30T04:38:00.754540Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:38:02.222944Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:38:04.359135Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:38:09.460657Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:39:45.907277Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:40:03.025600Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:40:23.814437Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:40:35.836493Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:40:39.470371Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:40:49.697223Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:40:54.095243Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T04:41:00.174Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 56 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "7a56f4ed-7fae-4b6a-8b76-990485437d67" "maas.apps.d36a3d177388305ff350.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.7:443 10.132.0.20:41550 - v1-completions-path 2026-10-30T04:41:12.280224Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:41:41.166539Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-10-30T04:41:41.090Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 77 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "4ce0a673-7401-47c6-a49d-84e804288218" "maas.apps.d36a3d177388305ff350.prod.konflux-ocp-ci.dev" "10.132.0.56:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.1.7:43698 10.134.1.7:443 10.132.0.20:47362 - v1-completions-path 2026-10-30T04:41:56.229382Z info ads ADS: new connection for node:4 2026-10-30T04:41:56.234634Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-10-30T04:41:56.234770Z info cache read certificate from file resource=file-root:system 2026-10-30T04:41:57.365687Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:41:58.901744Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:42:10.239654Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:42:10.239681Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:42:10.239693Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T04:42:10.239698Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:42:10.239844Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T04:42:10.126Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1720 113 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "cd43f95e-d813-48a7-9a89-ee7a5076dbdc" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.1.7:38610 10.134.1.7:443 10.132.0.20:34420 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T04:42:14.467027Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:42:14.467063Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:42:14.467076Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:42:14.467080Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T04:42:14.467245Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T04:42:14.364Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1724 102 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "0c0cb7ca-fed7-9234-bad8-9f50e8d04c4d" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.1.7:38610 10.134.1.7:443 10.132.0.20:34428 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T04:43:07.857841Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:43:07.857883Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:43:07.857901Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:43:07.857908Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T04:43:07.858131Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T04:43:07.757Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1728 101 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "01d2dcbf-c7e7-4694-a5f6-9bbfc2dbaeee" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.1.7:38610 10.134.1.7:443 10.132.0.20:40506 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T04:43:18.068780Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:43:18.068819Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:43:18.068835Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:43:18.068840Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T04:43:17.990Z] "DELETE /v1/api-keys/3b0ea34e-e043-4425-a56d-167c1176d31f HTTP/1.1" 200 - via_upstream - "-" 0 359 78 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "d9de27c3-662b-4aaf-9672-5301ffd802d9" "maas.apps.d36a3d177388305ff350.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.7:53138 10.134.1.7:443 10.132.0.20:43796 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T04:43:29.605663Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:43:29.731343Z info ads ADS: "" 4 terminated 2026-10-30T04:43:31.037745Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:43:36.635364Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:43:55.517545Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:44:00.089504Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:44:03.165566Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:44:03.165631Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:44:03.165645Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T04:44:03.165650Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T04:44:03.086Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 336 78 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "0fcbbbae-beff-96ef-b50e-af59e010fa3c" "maas.apps.d36a3d177388305ff350.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.7:35518 10.134.1.7:443 10.132.0.20:59334 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T04:44:15.446047Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:44:17.764805Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:44:20.719488Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:44:30.891389Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T04:44:37.809Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 72 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "5f0feebc-fff9-4903-a10c-958db06168dd" "maas.apps.d36a3d177388305ff350.prod.konflux-ocp-ci.dev" "10.132.0.55:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.7:35716 10.134.1.7:443 10.132.0.20:37596 - v1-completions-path 2026-10-30T04:44:54.383936Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:45:04.679041Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:45:09.152045Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:45:12.897496Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:45:12.897556Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:45:12.897614Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:45:12.897623Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T04:45:12.817Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 318 80 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "3c8e9a45-aa08-4379-91bf-e58a81570239" "maas.apps.d36a3d177388305ff350.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.7:54298 10.134.1.7:443 10.132.0.20:56678 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T04:45:13.094Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 64 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "f5b7fd34-b3ec-474e-8ac6-42fcfd778965" "maas.apps.d36a3d177388305ff350.prod.konflux-ocp-ci.dev" "10.132.0.55:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.7:35716 10.134.1.7:443 10.132.0.20:56694 - v1-completions-path [2026-10-30T04:45:14.060Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 54 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "4177890a-c60e-4e57-b699-d02eca344fdb" "maas.apps.d36a3d177388305ff350.prod.konflux-ocp-ci.dev" "10.132.0.55:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.7:35716 10.134.1.7:443 10.132.0.20:56706 - v1-completions-path 2026-10-30T04:45:26.316989Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:45:28.682529Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:45:47.795072Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:45:52.203376Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:46:07.692222Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:46:10.043726Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:46:27.646419Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:46:38.318012Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:46:41.175117Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:47:00.362640Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:47:03.329157Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T04:47:10.278Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 76 0 36 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "18366050-e9d2-4693-b4bc-c4342a725654" "maas.apps.d36a3d177388305ff350.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.7:443 10.132.0.20:37144 - v1-completions-path 2026-10-30T04:47:15.569901Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:47:21.127934Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:47:21.127976Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:47:21.127991Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:47:21.127997Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T04:47:21.053Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 65 308 74 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "65d328bd-3fad-4961-955b-34eb4cd9d803" "maas.apps.d36a3d177388305ff350.prod.konflux-ocp-ci.dev" "10.134.0.24:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.7:35678 10.134.1.7:443 10.132.0.20:36570 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T04:47:21.239Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 62 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "cb18db3f-bd7d-4340-ba6d-2bead28ad831" "maas.apps.d36a3d177388305ff350.prod.konflux-ocp-ci.dev" "10.132.0.55:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.7:35716 10.134.1.7:443 10.132.0.20:36582 - v1-completions-path 2026-10-30T04:47:33.379478Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:47:43.729035Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:48:03.019612Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:48:06.352314Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:49:48.064251Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:49:51.433929Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:51:21.508771Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:51:35.950385Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:51:38.312976Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:51:56.892699Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:52:08.457508Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:52:12.189446Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:52:16.342703Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:52:16.342746Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:52:16.342768Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T04:52:16.342774Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T04:52:16.269Z] "DELETE /v1/api-keys/36f1b2b4-e9de-4867-9531-170a41d2170a HTTP/1.1" 200 - via_upstream - "-" 0 369 73 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "f706824d-172b-4ad5-8424-fcd3d5b87a44" "maas.apps.d36a3d177388305ff350.prod.konflux-ocp-ci.dev" "10.134.1.25:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.7:45676 10.134.1.7:443 10.132.0.20:47038 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T04:52:27.383372Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:52:37.962492Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:52:40.529654Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:52:48.791000Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:52:48.791045Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:52:48.791064Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:52:48.791069Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T04:52:48.715Z] "DELETE /v1/api-keys/1c738fb5-7136-4371-acca-3c24cadccdd1 HTTP/1.1" 200 - via_upstream - "-" 0 365 76 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "a27d52e1-6ff8-41c5-b454-ae86514ec064" "maas.apps.d36a3d177388305ff350.prod.konflux-ocp-ci.dev" "10.132.1.69:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.7:37550 10.134.1.7:443 10.132.0.20:41064 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T04:53:00.711252Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:53:05.259141Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:53:08.224563Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T04:53:14.469Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 61 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "71099b79-3c15-4091-8474-b9e6744f6676" "maas.apps.d36a3d177388305ff350.prod.konflux-ocp-ci.dev" "10.132.0.55:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.7:60580 10.134.1.7:443 10.132.0.20:54792 - v1-completions-path 2026-10-30T04:53:16.762645Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-10-30T04:53:16.709Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 55 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "1c1f5d42-67e8-4795-8260-977bd223b2f1" "maas.apps.d36a3d177388305ff350.prod.konflux-ocp-ci.dev" "10.132.0.55:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.7:60580 10.134.1.7:443 10.132.0.20:54808 - v1-completions-path [2026-10-30T04:53:16.887Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 58 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "38ed37dd-7029-4cd5-9cae-18ce4c32cf0d" "maas.apps.d36a3d177388305ff350.prod.konflux-ocp-ci.dev" "10.132.0.55:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.7:60580 10.134.1.7:443 10.132.0.20:54816 - v1-completions-path [2026-10-30T04:53:17.426Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 55 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "c5923d6b-4fe6-4451-96b8-9cf4d8c340f3" "maas.apps.d36a3d177388305ff350.prod.konflux-ocp-ci.dev" "10.132.0.55:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.7:60580 10.134.1.7:443 10.132.0.20:54828 - v1-completions-path 2026-10-30T04:53:31.109097Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:53:41.471793Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:53:45.948529Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:53:48.988375Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:53:51.973253Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:53:51.973295Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:53:51.973312Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:53:51.973318Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T04:53:51.897Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 76 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "33d0178e-70a8-46dd-a65e-655c28bb7244" "maas.apps.d36a3d177388305ff350.prod.konflux-ocp-ci.dev" "10.132.1.69:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.7:59458 10.134.1.7:443 10.132.0.20:36430 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T04:53:54.028415Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:54:12.269508Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:54:22.483971Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:54:24.783616Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:54:34.262138Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:54:39.067563Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:54:55.535508Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:55:15.316945Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:55:26.552163Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:55:46.174390Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:55:56.641961Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:56:06.831303Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:56:18.138060Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:56:28.432197Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:56:38.793018Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:56:49.105565Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T04:57:00.192Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 289 75 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "63866033-db76-4d4c-8f2b-21b8981ab099" "maas.apps.d36a3d177388305ff350.prod.konflux-ocp-ci.dev" "10.132.0.56:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.1.7:43698 10.134.1.7:443 10.132.0.20:41856 - v1-completions-path