2026-10-30T03:53:20.710349Z info FLAG: --concurrency="0" 2026-10-30T03:53:20.710516Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T03:53:20.710541Z info FLAG: --help="false" 2026-10-30T03:53:20.710558Z info FLAG: --log_as_json="false" 2026-10-30T03:53:20.710574Z info FLAG: --log_caller="" 2026-10-30T03:53:20.710590Z info FLAG: --log_output_level="default:info" 2026-10-30T03:53:20.710606Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T03:53:20.710641Z info FLAG: --log_target="[stdout]" 2026-10-30T03:53:20.710657Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T03:53:20.710674Z info FLAG: --outlierLogPath="" 2026-10-30T03:53:20.710691Z info FLAG: --profiling="true" 2026-10-30T03:53:20.710707Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T03:53:20.710722Z info FLAG: --proxyLogLevel="warning" 2026-10-30T03:53:20.710738Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T03:53:20.710754Z info FLAG: --stsPort="0" 2026-10-30T03:53:20.710770Z info FLAG: --templateFile="" 2026-10-30T03:53:20.710785Z info FLAG: --tokenManagerPlugin="" 2026-10-30T03:53:20.710808Z info FLAG: --vklog="0" 2026-10-30T03:53:20.710825Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T03:53:20.710993Z info Proxy role ips=[10.134.1.11] type=router id=maas-default-gateway-openshift-default-645cf464d7-hj59z.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T03:53:20.711073Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T03:53:20.712181Z info cpu limit detected as 2, setting concurrency 2026-10-30T03:53:20.712432Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T03:53:20.712480Z info JWT policy is third-party-jwt 2026-10-30T03:53:20.712500Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T03:53:20.714501Z info Starting default Istio SDS Server 2026-10-30T03:53:20.714585Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T03:53:20.714651Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T03:53:20.715122Z info Opening status port 15020 2026-10-30T03:53:20.715739Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T03:53:20.716749Z info sds Starting SDS grpc server 2026-10-30T03:53:20.716769Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T03:53:20.717500Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T03:53:20.718656Z info Starting proxy agent 2026-10-30T03:53:20.718696Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T03:53:20.805207Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T03:53:20.814129Z info cache generated new workload certificate resourceName=default latency=97.150532ms ttl=23h59m59.185874837s 2026-10-30T03:53:20.814154Z info cache Root cert has changed, start rotating root cert 2026-10-30T03:53:20.814196Z info cache returned workload trust anchor from cache ttl=23h59m59.18580427s 2026-10-30T03:53:20.922158Z info ads ADS: new connection for node:1 2026-10-30T03:53:20.922368Z info cache returned workload certificate from cache ttl=23h59m59.077635505s 2026-10-30T03:53:20.923412Z info ads ADS: new connection for node:2 2026-10-30T03:53:20.923638Z info cache returned workload trust anchor from cache ttl=23h59m59.076364556s 2026-10-30T03:53:21.017251Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:53:29.379624Z info Readiness succeeded in 8.672776702s 2026-10-30T03:53:29.380422Z info Envoy proxy is ready 2026-10-30T03:53:49.935740Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T03:53:54.147Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 140 405 85 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "4913e018-ce4d-4b6d-8dac-a3014bdef570" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.35:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.134.1.11:53418 10.134.1.11:443 10.132.0.13:38224 - v1-chat-completions-path 2026-10-30T03:54:06.152287Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:54:08.767340Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:54:16.039738Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:54:16.039775Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T03:54:16.039790Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T03:54:16.039795Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T03:54:15.961Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 323 78 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "717d6588-5659-4d6d-a18a-cdea128a0626" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.11:59598 10.134.1.11:443 10.132.0.13:36146 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T03:54:19.275636Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T03:54:23.027Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 429 - - "-" 61 18 66 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "e5f34a23-7686-4759-806a-a646f67aa700" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.11:443 10.132.0.13:36170 - v1-catch-all-path 2026-10-30T03:54:35.305199Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:54:37.707204Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:54:52.423988Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:55:03.552180Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:55:05.237251Z info ads ADS: new connection for node:3 2026-10-30T03:55:05.237478Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T03:55:05.237513Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T03:55:07.516863Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:55:49.706239Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T03:55:54.759Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 329 94 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "7caa64e7-162e-403b-807e-135fba90996d" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.1.67:8000" outbound|54321||e2e-prio-pool-0e6257a1-inference-pool-ip-0cbeffb6.llm.svc.cluster.local 10.134.1.11:53124 10.134.1.11:443 10.132.0.13:60630 - v1-model-routing 2026-10-30T03:56:08.988910Z info ads ADS: "" 3 terminated 2026-10-30T03:56:09.036830Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:56:11.386507Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:56:14.263271Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:56:19.470988Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:57:55.593585Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:58:13.230250Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:58:33.660372Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:58:45.682950Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:58:50.578824Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:59:03.294286Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:59:06.954247Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T03:59:26.895432Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:00:13.985101Z info ads ADS: new connection for node:4 2026-10-30T04:00:13.990558Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-10-30T04:00:13.990612Z info cache read certificate from file resource=file-root:system 2026-10-30T04:00:15.165573Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:00:15.953636Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:00:19.034683Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T04:00:19.740Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 429 - - "-" 92 18 91 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "316c652e-0c76-44c1-9813-63f4aec56ca9" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.134.1.11:443 10.132.0.13:59062 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-10-30T04:00:39.049423Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:00:39.049458Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:00:39.049469Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T04:00:39.049473Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:00:39.049650Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T04:00:38.936Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1725 113 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "47f51dea-e78c-4ca8-9bb3-f1e386a49290" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.1.11:41466 10.134.1.11:443 10.132.0.13:33122 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T04:00:51.844997Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:00:51.845062Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:00:51.845081Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:00:51.845088Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T04:00:51.845272Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-10-30T04:00:51.845317Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-10-30T04:00:51.739Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1728 105 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "75039fb6-5338-4544-a100-ce27e93fa31a" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.1.11:51880 10.134.1.11:443 10.132.0.13:32956 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T04:01:21.742375Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:01:21.742409Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:01:21.742421Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:01:21.742425Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T04:01:21.742589Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-10-30T04:01:21.640Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1732 102 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "b426b0e4-bd21-496c-8656-4b37c84300dd" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.1.11:51880 10.134.1.11:443 10.132.0.13:54576 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T04:01:23.873380Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:01:23.873411Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:01:23.873419Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T04:01:23.873424Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:01:23.873569Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T04:01:23.766Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1728 106 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "e130ebb1-5024-4ee0-a36c-4288bda166e8" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.1.11:41466 10.134.1.11:443 10.132.0.13:54582 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T04:01:30.271966Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:01:30.272000Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:01:30.272012Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:01:30.272016Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T04:01:30.272214Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-10-30T04:01:30.173Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1727 98 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "46eeae77-f50e-443d-9e53-100deb49cc96" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.1.11:51880 10.134.1.11:443 10.132.0.13:32768 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T04:01:56.540072Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:01:56.651612Z info ads ADS: "" 4 terminated with stream closed 2026-10-30T04:01:57.231701Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:01:59.320990Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:02:06.328301Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:02:10.244367Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-10-30T04:02:10.254246Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:02:10.254280Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:02:10.254295Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:02:10.254300Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T04:02:10.173Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 87 329 80 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "ac212b32-699a-490b-bb77-b0e989f07e3c" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.11:60182 10.134.1.11:443 10.132.0.13:47356 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T04:02:10.475Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 74 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "d7f3f7c0-adaf-4e1e-ad25-fc6e1dca7dbd" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.11:43178 10.134.1.11:443 10.132.0.13:47366 - v1-completions-path [2026-10-30T04:02:11.284Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 57 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "c7fa3d99-cc76-4958-b8cc-2b944b38d577" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.11:443 10.132.0.13:47368 - v1-completions-path 2026-10-30T04:02:23.276328Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:02:27.949518Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:02:31.055151Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:02:31.055186Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:02:31.055201Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:02:31.055206Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T04:02:30.980Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 336 74 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "5b4351f2-f517-45f3-b552-c57386a64ca1" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.11:60182 10.134.1.11:443 10.132.0.13:60764 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T04:02:43.422066Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:02:59.199283Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:03:04.334145Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:03:04.334181Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:03:04.334196Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:03:04.334201Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T04:03:04.249Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 323 84 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "e5a75059-adbe-493b-9927-efce2d8abd56" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.11:60182 10.134.1.11:443 10.132.0.13:48290 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T04:03:05.859Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 69 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "917b05a0-294d-48f0-a8b5-c461d0de50e4" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.11:43178 10.134.1.11:443 10.132.0.13:43980 - v1-completions-path [2026-10-30T04:03:07.257Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 298 60 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "8c74f366-a342-4594-bfb2-0398a6268c9f" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.11:43178 10.134.1.11:443 10.132.0.13:43986 - v1-completions-path 2026-10-30T04:03:22.605829Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:03:33.055268Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:03:37.536600Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:03:41.454742Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:03:41.454784Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:03:41.454807Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T04:03:41.454813Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T04:03:41.370Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 317 84 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "24b4ef03-d485-4b95-bcb4-5f740bf5bcd2" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.11:60494 10.134.1.11:443 10.132.0.13:44726 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T04:03:41.991Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 55 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "4dd681c7-536f-4982-bc50-96d0b3b7170b" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.11:43178 10.134.1.11:443 10.132.0.13:44736 - v1-completions-path [2026-10-30T04:03:42.144Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 60 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "3446f887-ed1c-404d-9fc7-a8faa27733fc" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.11:43574 10.134.1.11:443 10.132.0.13:44738 - v1-completions-path [2026-10-30T04:03:42.647Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 57 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "85b17fa7-5dd1-4062-a602-58d820d8902d" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.11:43574 10.134.1.11:443 10.132.0.13:44752 - v1-completions-path 2026-10-30T04:03:54.750285Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:03:57.042651Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:04:16.475108Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:04:21.226421Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:04:36.977862Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:04:39.352252Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:04:57.111593Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:05:07.951535Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:05:10.941350Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T04:05:16.381Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 299 79 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "ad3c3081-d77f-4cf1-be15-695870025c5f" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.36:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.1.11:33180 10.134.1.11:443 10.132.0.13:59398 - v1-completions-path 2026-10-30T04:05:29.180063Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:05:32.102260Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T04:05:39.422Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 76 0 35 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "357d97c3-8bb9-4861-ac82-f0d148eb2b17" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.11:443 10.132.0.13:42978 - v1-completions-path 2026-10-30T04:05:44.092158Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:06:03.030991Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:06:13.370158Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:06:20.948865Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:06:20.948901Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:06:20.948918Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T04:06:20.948923Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T04:06:20.869Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 79 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "657dc28d-951d-41b4-92f0-9808edfe0694" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.11:59096 10.134.1.11:443 10.132.0.13:42306 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T04:06:33.673163Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:06:37.241146Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:08:08.029188Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:08:08.029225Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:08:08.029241Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T04:08:08.029245Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T04:08:07.937Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 80 323 91 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "b39dbc5f-7192-4ba2-9740-e91b2c133da7" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.11:44536 10.134.1.11:443 10.132.0.13:36786 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T04:08:18.613Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 49 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "b7755ed6-535c-44c2-8f22-9645e55576c0" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.11:443 10.132.0.13:48188 - v1-completions-path 2026-10-30T04:08:20.844687Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:08:24.268508Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:09:35.456926Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:09:35.456967Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:09:35.456982Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T04:09:35.456987Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T04:09:35.380Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 76 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "a9647711-0139-4afa-8ba8-c6703b60b95d" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.11:39850 10.134.1.11:443 10.132.0.13:35100 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T04:09:54.539052Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:10:09.939193Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:10:12.304751Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:10:31.368567Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:10:43.690081Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:10:47.289825Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T04:10:50.516Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 65 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "4a39ace3-423d-467d-8563-21ce75356ed7" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.11:43178 10.134.1.11:443 10.132.0.13:55518 - v1-completions-path 2026-10-30T04:10:51.566176Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:10:51.566212Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:10:51.566227Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:10:51.566232Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T04:10:50.705Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 60 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "0c2cdee7-568d-4a4a-bf86-966992c73725" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.11:43574 10.134.1.11:443 10.132.0.13:55526 - v1-completions-path [2026-10-30T04:10:51.492Z] "DELETE /v1/api-keys/134e198f-3d72-44eb-97dc-a92cd706011c HTTP/1.1" 200 - via_upstream - "-" 0 369 74 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "b7f086f9-41ab-4b48-803c-13576c9ca1a2" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.1.108:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.11:45228 10.134.1.11:443 10.132.0.13:55540 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T04:11:03.175532Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:11:13.412849Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:11:18.578759Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T04:11:22.837Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 59 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "69ee19a8-e02d-46a2-b159-d26d87bca6f5" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.11:43178 10.134.1.11:443 10.132.0.13:39780 - v1-completions-path 2026-10-30T04:11:24.017702Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:11:24.017743Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:11:24.017759Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T04:11:24.017764Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T04:11:23.933Z] "DELETE /v1/api-keys/e130c46a-d6a9-4351-b41d-c5122fc83f6a HTTP/1.1" 200 - via_upstream - "-" 0 365 84 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "69b5168f-6a2f-4abb-8ca9-1127c7beaa15" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.132.1.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.11:55810 10.134.1.11:443 10.132.0.13:39794 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T04:11:36.157468Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:11:40.563549Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:11:44.187597Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:11:50.321576Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:11:50.321616Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:11:50.321633Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T04:11:50.321638Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T04:11:50.240Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 68 310 81 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "06b5e5e3-e719-4c45-b698-c92204c11493" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.1.108:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.11:36468 10.134.1.11:443 10.132.0.13:35650 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T04:11:51.271Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 60 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "9b6dd985-a73e-4c33-b82a-ee5f89441c98" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.11:443 10.132.0.13:35660 - v1-completions-path [2026-10-30T04:11:54.647Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 61 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "9b46a813-9181-4739-ba19-0819ea25a21e" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.11:43574 10.134.1.11:443 10.132.0.13:50076 - v1-completions-path 2026-10-30T04:12:07.228471Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:12:17.567797Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:12:22.168456Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:12:25.619941Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:12:30.564432Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T04:12:35.843Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 62 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "21846d86-4806-4992-83ff-81ccbfb4f35d" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.11:43178 10.134.1.11:443 10.132.0.13:53344 - v1-completions-path [2026-10-30T04:12:36.210Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 64 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "f14427c5-467a-47af-ac3a-6d2ffc4a8dfc" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.11:43574 10.134.1.11:443 10.132.0.13:53352 - v1-completions-path 2026-10-30T04:12:48.911833Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:12:59.247672Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:13:01.533490Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:13:11.278200Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:13:16.381694Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:13:33.124238Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:13:53.266853Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:14:03.545704Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:14:13.835661Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:14:24.282479Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:14:34.557342Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:14:44.991073Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:14:55.565950Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:15:05.848765Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:15:16.176282Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:15:26.519059Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T04:15:36.037770Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T04:15:36.037796Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T04:15:36.037800Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T04:15:36.037804Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-10-30T04:15:36.007Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 401 - - "-" 66 88 30 - "34.228.250.194,10.132.0.13" "python-requests/2.34.2" "7b6dca6d-70a4-40ae-8ce5-738327828a21" "maas.apps.de2b76ef3c818f346074.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.11:443 10.132.0.13:59696 - v1-completions-path