2026-10-30T07:22:15.733424Z info FLAG: --concurrency="0" 2026-10-30T07:22:15.733448Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T07:22:15.733452Z info FLAG: --help="false" 2026-10-30T07:22:15.733455Z info FLAG: --log_as_json="false" 2026-10-30T07:22:15.733457Z info FLAG: --log_caller="" 2026-10-30T07:22:15.733459Z info FLAG: --log_output_level="default:info" 2026-10-30T07:22:15.733461Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T07:22:15.733466Z info FLAG: --log_target="[stdout]" 2026-10-30T07:22:15.733468Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T07:22:15.733470Z info FLAG: --outlierLogPath="" 2026-10-30T07:22:15.733472Z info FLAG: --profiling="true" 2026-10-30T07:22:15.733475Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T07:22:15.733477Z info FLAG: --proxyLogLevel="warning" 2026-10-30T07:22:15.733479Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T07:22:15.733483Z info FLAG: --stsPort="0" 2026-10-30T07:22:15.733484Z info FLAG: --templateFile="" 2026-10-30T07:22:15.733486Z info FLAG: --tokenManagerPlugin="" 2026-10-30T07:22:15.733489Z info FLAG: --vklog="0" 2026-10-30T07:22:15.733492Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T07:22:15.733664Z info Proxy role ips=[10.133.0.252] type=router id=maas-default-gateway-openshift-default-85544d4cd7-s7qrv.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T07:22:15.733708Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T07:22:15.735294Z info cpu limit detected as 2, setting concurrency 2026-10-30T07:22:15.735597Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T07:22:15.735610Z info JWT policy is third-party-jwt 2026-10-30T07:22:15.735614Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T07:22:15.737218Z info Starting default Istio SDS Server 2026-10-30T07:22:15.737242Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T07:22:15.737259Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T07:22:15.737261Z info Opening status port 15020 2026-10-30T07:22:15.738381Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T07:22:15.742296Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T07:22:15.743564Z info Starting proxy agent 2026-10-30T07:22:15.743607Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T07:22:15.756850Z info sds Starting SDS grpc server 2026-10-30T07:22:15.756898Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T07:22:15.812066Z info cache generated new workload certificate resourceName=default latency=73.635931ms ttl=23h59m59.187938133s 2026-10-30T07:22:15.812113Z info cache Root cert has changed, start rotating root cert 2026-10-30T07:22:15.812156Z info cache returned workload trust anchor from cache ttl=23h59m59.187844596s 2026-10-30T07:22:15.819412Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T07:22:16.050222Z info ads ADS: new connection for node:1 2026-10-30T07:22:16.050296Z info cache returned workload certificate from cache ttl=23h59m58.94970689s 2026-10-30T07:22:16.050673Z info ads ADS: new connection for node:2 2026-10-30T07:22:16.050742Z info cache returned workload trust anchor from cache ttl=23h59m58.949259049s 2026-10-30T07:22:16.148435Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:22:24.385766Z info Readiness succeeded in 8.655679348s 2026-10-30T07:22:24.386227Z info Envoy proxy is ready 2026-10-30T07:23:32.865997Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:23:53.872544Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:25:37.091646Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:27:11.546619Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-10-30T07:34:09.781116Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:34:33.805163Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:34:44.766599Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T07:34:44.766636Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T07:34:44.766648Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T07:34:44.766654Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T07:34:44.686Z] "POST /v1/api-keys HTTP/1.1" 400 - via_upstream - "-" 80 94 80 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "0c61a57c-f995-4cde-8ad6-0832a8773377" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.252:49358 10.133.0.252:443 10.132.0.21:60324 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T07:35:08.368813Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T07:35:13.682Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 2817 3089 79 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "7a63bd22-244a-496a-9022-72c666acdb6a" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.133.0.36:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.133.0.252:37116 10.133.0.252:443 10.132.0.21:41204 - v1-chat-completions-path [2026-10-30T07:35:14.064Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 108118 108394 176 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "3cf65a8f-c624-41f6-93ed-54171163ae14" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.133.0.36:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.133.0.252:37130 10.133.0.252:443 10.132.0.21:41222 - v1-chat-completions-path [2026-10-30T07:35:14.265Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 144 409 58 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "5add51ad-e7b5-46bb-a188-5fcfb9d78311" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.133.0.36:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.133.0.252:37130 10.133.0.252:443 10.132.0.21:41224 - v1-chat-completions-path 2026-10-30T07:35:26.519200Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:35:28.776581Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:35:39.655301Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:35:45.280929Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-10-30T07:35:45.205Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 75 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "adc54f65-0da0-4057-8a3e-c1b9a2977a7b" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.32:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.252:45424 10.133.0.252:443 10.132.0.21:38976 - v1-catch-all-path 2026-10-30T07:35:57.712609Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:36:00.014489Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:36:14.816358Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:36:25.834519Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:36:27.970421Z info ads ADS: new connection for node:3 2026-10-30T07:36:27.970633Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T07:36:27.970667Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T07:36:29.882592Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:37:10.393249Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:37:15.874419Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-10-30T07:37:15.420Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 340 90 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "ec166ce2-f9a0-4053-b139-234db06b965c" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.133.1.21:8000" outbound|54321||e2e-prio-pool-38cb65a5-inference-pool-ip-555586cf.llm.svc.cluster.local 10.133.0.252:33962 10.133.0.252:443 10.132.0.21:58518 - v1-model-routing [2026-10-30T07:37:15.816Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 333 61 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "4c6c16ab-d2be-4c5d-9248-ebc6fbe94634" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.133.1.21:8000" outbound|54321||e2e-prio-pool-38cb65a5-inference-pool-ip-555586cf.llm.svc.cluster.local 10.133.0.252:33962 10.133.0.252:443 10.132.0.21:58526 - v1-model-routing 2026-10-30T07:37:29.360113Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:37:29.449282Z info ads ADS: "" 3 terminated with stream closed 2026-10-30T07:37:32.095276Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-10-30T07:37:32.024Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 290 71 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "788af54e-9cf8-4a9d-92dc-0a5688c72807" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.252:36036 10.133.0.252:443 10.132.0.21:42160 - v1-completions-path 2026-10-30T07:37:33.028266Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:37:40.622683Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:39:08.760019Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T07:39:08.760072Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T07:39:08.760096Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T07:39:08.760102Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T07:39:08.684Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 75 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "2bb23b8d-28fd-4bf2-a530-d06582adce95" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.252:37790 10.133.0.252:443 10.132.0.21:60514 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T07:39:16.049784Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:39:32.464547Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:39:47.846304Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:40:01.890659Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:40:06.749439Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:40:19.300311Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:40:22.895230Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:40:43.024984Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:41:30.290731Z info ads ADS: new connection for node:4 2026-10-30T07:41:30.296572Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-10-30T07:41:30.296629Z info cache read certificate from file resource=file-root:system 2026-10-30T07:41:31.421702Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:41:32.997512Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:42:31.056941Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:42:31.056978Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:42:31.056993Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:42:31.056997Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T07:42:31.057198Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T07:42:30.856Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1722 200 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "215e4bbd-812f-4e11-8adc-017cdf363411" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.252:40818 10.133.0.252:443 10.132.0.21:59478 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T07:42:43.830908Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:42:43.830944Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:42:43.830957Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:42:43.830962Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T07:42:43.831157Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T07:42:43.732Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1726 98 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "ba945a89-c8df-4a54-a5e6-5cd9ca8d7dac" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.252:40818 10.133.0.252:443 10.132.0.21:47888 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T07:42:59.296686Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:42:59.451594Z info ads ADS: "" 4 terminated with stream closed 2026-10-30T07:43:00.883305Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:43:07.784221Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T07:43:12.500Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 77 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "89a1efe5-109b-4b7a-aa04-0d08cac0cf8b" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.252:34628 10.133.0.252:443 10.132.0.21:60484 - v1-completions-path 2026-10-30T07:43:24.965358Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:43:29.433377Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T07:43:33.088Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 59 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "6c69ef77-7743-407e-9c71-4830615ff866" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.252:443 10.132.0.21:42560 - v1-completions-path 2026-10-30T07:43:45.199529Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:44:01.018860Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T07:44:06.267Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 65 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "f8a93878-273e-411b-8708-b412963c5b66" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.252:59338 10.133.0.252:443 10.132.0.21:43504 - v1-completions-path [2026-10-30T07:44:06.354Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 54 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "78b1a478-e351-4ef8-900e-a8c50440186b" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.252:59338 10.133.0.252:443 10.132.0.21:43514 - v1-completions-path [2026-10-30T07:44:06.854Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 58 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "748bc013-3d1d-4975-b345-477cd1362d20" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.252:34628 10.133.0.252:443 10.132.0.21:43520 - v1-completions-path [2026-10-30T07:44:07.248Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 115 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "b2077bba-5fdf-4e0d-895b-fe4090851eff" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.252:34628 10.133.0.252:443 10.132.0.21:43528 - v1-completions-path 2026-10-30T07:44:23.610920Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:44:33.849893Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:44:38.716930Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T07:44:45.535Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 62 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "65ef3ee5-b220-46bc-8382-88ae8d298e07" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.252:34628 10.133.0.252:443 10.132.0.21:50056 - v1-completions-path [2026-10-30T07:44:46.344Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 59 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "62f41106-7f4d-4eda-bfb6-f6eb2b10c68f" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.252:59338 10.133.0.252:443 10.132.0.21:50058 - v1-completions-path 2026-10-30T07:44:58.721087Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:45:01.050744Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:45:08.302454Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:45:08.302493Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:45:08.302508Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:45:08.302526Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T07:45:08.228Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 73 316 73 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "9a38fa2b-8b1d-4a44-9a77-210dcebe4ae8" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.252:57854 10.133.0.252:443 10.132.0.21:36968 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T07:45:20.570828Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:45:22.922574Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:45:41.539566Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:45:43.846012Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:46:01.709040Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:46:12.394857Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:46:15.284583Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:46:21.800796Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T07:46:21.800841Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T07:46:21.800863Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T07:46:21.800896Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T07:46:21.724Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 323 76 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "4d5593a4-3138-4d36-998a-23418f860770" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.252:49128 10.133.0.252:443 10.132.0.21:42444 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T07:46:34.595188Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:46:37.422810Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:46:49.998698Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:47:07.981123Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:47:18.341722Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:47:38.402035Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:47:41.784790Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:49:12.152648Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:49:12.152686Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:49:12.152701Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:49:12.152706Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T07:49:12.081Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 80 322 71 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "d0e9d4b9-07e5-4cf3-bbb5-058a2aca5039" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.252:42332 10.133.0.252:443 10.132.0.21:47262 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T07:49:24.888933Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:49:28.380273Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:51:01.742854Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:51:17.260534Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:51:19.739951Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:51:38.592256Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:51:50.591605Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:51:53.553744Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:52:09.441116Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:52:19.992312Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:52:24.648866Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:52:41.870364Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:52:46.495045Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:52:49.540443Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T07:52:56.369Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 58 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "71ba2ee1-6448-44ea-a668-139a782e60e8" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.252:34628 10.133.0.252:443 10.132.0.21:36366 - v1-completions-path 2026-10-30T07:53:12.529969Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:53:22.947283Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:53:27.475353Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:53:31.130280Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:53:33.839339Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T07:53:33.839377Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T07:53:33.839393Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T07:53:33.839397Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T07:53:33.713Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 126 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "bc259d86-e154-4ef9-8223-d93a50f7ff48" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.133.1.61:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.252:42622 10.133.0.252:443 10.132.0.21:58962 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T07:53:36.185188Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:53:54.322204Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:54:04.741002Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:54:07.090762Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:54:18.448158Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:54:22.099583Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:54:39.765006Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:54:42.150704Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:55:00.451639Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:55:10.925603Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:55:21.066858Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:55:31.443159Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:55:41.842324Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:55:52.341666Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:56:02.510112Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:56:12.951863Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:56:23.149420Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:56:33.493021Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T07:56:36.138Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 72 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "717fb2c3-6e71-4073-b3cd-5824d4f829b7" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.252:51550 10.133.0.252:443 10.132.0.21:52014 - v1-completions-path [2026-10-30T07:56:40.645Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 43 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "5f6a00e4-884b-4bae-9dce-af3e2457a389" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.252:443 10.132.0.21:54980 - v1-completions-path