2026-10-30T13:52:48.152804Z info FLAG: --concurrency="0" 2026-10-30T13:52:48.152827Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T13:52:48.152831Z info FLAG: --help="false" 2026-10-30T13:52:48.152833Z info FLAG: --log_as_json="false" 2026-10-30T13:52:48.152835Z info FLAG: --log_caller="" 2026-10-30T13:52:48.152837Z info FLAG: --log_output_level="default:info" 2026-10-30T13:52:48.152839Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T13:52:48.152843Z info FLAG: --log_target="[stdout]" 2026-10-30T13:52:48.152846Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T13:52:48.152847Z info FLAG: --outlierLogPath="" 2026-10-30T13:52:48.152849Z info FLAG: --profiling="true" 2026-10-30T13:52:48.152851Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T13:52:48.152853Z info FLAG: --proxyLogLevel="warning" 2026-10-30T13:52:48.152855Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T13:52:48.152857Z info FLAG: --stsPort="0" 2026-10-30T13:52:48.152859Z info FLAG: --templateFile="" 2026-10-30T13:52:48.152860Z info FLAG: --tokenManagerPlugin="" 2026-10-30T13:52:48.152863Z info FLAG: --vklog="0" 2026-10-30T13:52:48.152865Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T13:52:48.153027Z info Proxy role ips=[10.132.0.30] type=router id=maas-default-gateway-openshift-default-fd4b99879-2vmwg.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T13:52:48.153103Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T13:52:48.154195Z info cpu limit detected as 2, setting concurrency 2026-10-30T13:52:48.154397Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T13:52:48.154407Z info JWT policy is third-party-jwt 2026-10-30T13:52:48.154410Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T13:52:48.156338Z info Starting default Istio SDS Server 2026-10-30T13:52:48.156371Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T13:52:48.156395Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T13:52:48.156715Z info Opening status port 15020 2026-10-30T13:52:48.157777Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T13:52:48.159171Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T13:52:48.160322Z info Starting proxy agent 2026-10-30T13:52:48.160365Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T13:52:48.168725Z info sds Starting SDS grpc server 2026-10-30T13:52:48.168747Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T13:52:48.226353Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T13:52:48.263719Z info cache generated new workload certificate resourceName=default latency=106.237618ms ttl=23h59m59.736284493s 2026-10-30T13:52:48.263754Z info cache Root cert has changed, start rotating root cert 2026-10-30T13:52:48.263789Z info cache returned workload trust anchor from cache ttl=23h59m59.73621165s 2026-10-30T13:52:48.352467Z info ads ADS: new connection for node:1 2026-10-30T13:52:48.352840Z info cache returned workload certificate from cache ttl=23h59m59.647166909s 2026-10-30T13:52:48.352987Z info ads ADS: new connection for node:2 2026-10-30T13:52:48.353357Z info cache returned workload trust anchor from cache ttl=23h59m59.646645372s 2026-10-30T13:52:48.444068Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:52:56.182498Z info Readiness succeeded in 8.032666697s 2026-10-30T13:52:56.182944Z info Envoy proxy is ready 2026-10-30T13:53:14.828937Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:53:32.432628Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T13:53:32.432659Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T13:53:32.432664Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T13:53:32.432670Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-10-30T13:53:32.356Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 403 - - "-" 61 103 75 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "58900426-3609-4def-9c45-82dbc8525ce2" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.30:443 10.132.0.12:48754 - v1-catch-all-path 2026-10-30T13:53:33.127428Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:53:36.083268Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:53:46.151563Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:53:51.847594Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-10-30T13:53:51.763Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 83 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "8b14f80b-4d41-4de3-80f8-9c9c35510cf5" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.133.0.36:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.30:33196 10.132.0.30:443 10.132.0.12:37160 - v1-catch-all-path 2026-10-30T13:54:04.220792Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:54:06.627806Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:54:21.733449Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:54:32.548534Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:54:34.391128Z info ads ADS: new connection for node:3 2026-10-30T13:54:34.391335Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T13:54:34.391374Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T13:54:36.384360Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:55:16.807672Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T13:55:22.057Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 349 85 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "5965dfc5-2c9c-4d7d-805d-435556a30b5a" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.133.1.40:8000" outbound|54321||e2e-prio-pool-47dd2769-inference-pool-ip-de055579.llm.svc.cluster.local 10.132.0.30:58258 10.132.0.30:443 10.132.0.12:36592 - v1-model-routing [2026-10-30T13:55:22.171Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 330 63 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "95f3da33-705b-49ca-8c0c-106f7fb7b88c" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.133.1.40:8000" outbound|54321||e2e-prio-pool-47dd2769-inference-pool-ip-de055579.llm.svc.cluster.local 10.132.0.30:58260 10.132.0.30:443 10.132.0.12:36606 - v1-model-routing 2026-10-30T13:55:35.811514Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:55:35.942679Z info ads ADS: "" 3 terminated with stream closed 2026-10-30T13:55:38.978428Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:55:46.769949Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:57:17.029227Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T13:57:17.029263Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T13:57:17.029275Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T13:57:17.029279Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T13:57:16.956Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 73 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "22cbe358-5818-4aa1-8b6d-4245ae30d45d" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.133.0.28:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.30:47910 10.132.0.30:443 10.132.0.12:50978 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T13:57:22.985632Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:57:39.201123Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:57:54.229589Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:58:08.266029Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:58:12.088616Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:58:23.803353Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:58:27.814796Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T13:58:33.988Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 61 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "1f376c24-23f9-4d10-9753-e944c409a341" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.30:443 10.132.0.12:35136 - v1-completions-path 2026-10-30T13:58:46.101160Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:58:48.709488Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-10-30T13:58:48.709575Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T13:58:48.709597Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T13:58:48.709616Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T13:58:48.709620Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T13:58:48.629Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 324 80 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "953cc189-ea68-4913-b70f-c5efe101c5bc" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.133.0.28:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.30:36470 10.132.0.30:443 10.132.0.12:41204 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T13:59:02.664Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 299 79 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "981ef05e-3348-4c00-9d5b-2b47961ce1d4" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.133.0.38:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.30:40520 10.132.0.30:443 10.132.0.12:55940 - v1-completions-path 2026-10-30T13:59:32.534638Z info ads ADS: new connection for node:4 2026-10-30T13:59:32.547219Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-10-30T13:59:32.547280Z info cache read certificate from file resource=file-root:system 2026-10-30T13:59:33.569845Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:59:35.405483Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T13:59:36.071298Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"142","StreamId":"2947837565252277732"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=24 [2026-10-30T13:59:36.050Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 21 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "018fc0a2-63ed-4d9f-85c4-54127be39f07" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.132.0.30:443 10.132.0.12:36608 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-10-30T13:59:40.481337Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"146","StreamId":"4345216831263415231"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=25 [2026-10-30T13:59:40.421Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 60 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "39ba3dc4-019d-49f1-a700-89f276579d9d" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.132.0.30:443 10.132.0.12:60300 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-10-30T14:00:35.645335Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:00:35.645373Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:00:35.645383Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:00:35.645387Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T14:00:35.645557Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-10-30T14:00:35.535Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1718 110 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "2bc0cd44-04c5-42d0-a9bd-eac6c546f8f4" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.30:53522 10.132.0.30:443 10.132.0.12:41582 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T14:00:37.773069Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:00:37.773103Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:00:37.773114Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T14:00:37.773118Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:00:37.773273Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-10-30T14:00:37.676Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1712 96 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "979be9ad-8058-4eb9-a728-55b655425f7b" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.30:53522 10.132.0.30:443 10.132.0.12:59054 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T14:00:58.925092Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-10-30T14:00:59.018818Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:00:59.018860Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:00:59.018894Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:00:59.018898Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T14:00:59.019099Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-10-30T14:00:58.917Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1720 101 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "2017a388-ba6c-41b3-bdb1-29be11cfd740" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.30:53522 10.132.0.30:443 10.132.0.12:36388 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T14:01:05.388896Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:01:05.388937Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:01:05.388949Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:01:05.388953Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T14:01:05.389138Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-10-30T14:01:05.293Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1718 95 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "5051c066-977d-4b14-9ded-d491cbcd4f0d" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.30:53522 10.132.0.30:443 10.132.0.12:36408 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T14:01:16.047123Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T14:01:16.047160Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T14:01:16.047172Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T14:01:16.047176Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T14:01:16.047381Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T14:01:15.938Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1717 108 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "a2a64c12-7147-4635-855f-b9710a699f4c" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.30:55762 10.132.0.30:443 10.132.0.12:57764 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T14:01:29.319777Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:01:29.412526Z info ads ADS: "" 4 terminated 2026-10-30T14:01:30.816328Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:01:37.691780Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:01:54.612489Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:01:59.141042Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:02:17.536863Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:02:33.469791Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T14:02:43.743Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 76 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "2adf769a-2658-40f7-9d82-aaacb45b97cb" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.30:40376 10.132.0.30:443 10.132.0.12:42272 - v1-completions-path [2026-10-30T14:02:45.845Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 58 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "bd00c987-3a7b-4a16-834e-1090b5031fe8" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.30:40386 10.132.0.30:443 10.132.0.12:42274 - v1-completions-path 2026-10-30T14:03:02.479957Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:03:12.816575Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:03:17.382273Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T14:03:24.393Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 64 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "b92a722a-c173-4b41-89db-ff6cc4a53817" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.30:40386 10.132.0.30:443 10.132.0.12:35316 - v1-completions-path [2026-10-30T14:03:25.639Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 59 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "b1417561-4706-4daf-b359-03fd7051155a" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.30:40376 10.132.0.30:443 10.132.0.12:35340 - v1-completions-path 2026-10-30T14:03:37.652372Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:03:40.074676Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T14:03:47.004Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 74 288 84 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "e28b448f-599c-4d49-bfbf-6c26dd7e97df" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.132.0.47:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.132.0.30:53416 10.132.0.30:443 10.132.0.12:56798 - v1-completions-path 2026-10-30T14:03:59.089331Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:04:03.646581Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T14:04:20.468Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 283 73 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "800f77d6-ae06-4064-a07d-25ae6f317d70" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.133.0.38:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.30:34186 10.132.0.30:443 10.132.0.12:47216 - v1-completions-path 2026-10-30T14:04:21.985705Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:04:24.272900Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:04:41.951189Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:04:53.602625Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:04:56.674472Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:05:17.089744Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:05:19.373685Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:05:32.951925Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:05:38.277926Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T14:05:38.277961Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T14:05:38.277976Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T14:05:38.277981Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T14:05:38.197Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 67 310 80 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "8aa904e5-592b-4187-b10f-f5f95ba025f0" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.133.0.28:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.30:37846 10.132.0.30:443 10.132.0.12:39394 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T14:05:50.747333Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:06:01.018398Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T14:06:18.300Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 36 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "a72a18cd-5a07-46cb-8962-aac2691928b9" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.30:443 10.132.0.12:41990 - v1-completions-path 2026-10-30T14:06:20.570124Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:06:24.059556Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:08:08.053342Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:08:11.653301Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:09:46.260683Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:10:01.734231Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:10:04.110046Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:10:10.830258Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T14:10:10.830286Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T14:10:10.830291Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T14:10:10.830297Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-10-30T14:10:10.797Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 123 33 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "6e57bb3e-6c79-4c87-a293-6641b69a7764" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.30:443 10.132.0.12:49220 - v1-completions-path 2026-10-30T14:10:22.801718Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:10:34.523229Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:10:37.755475Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:10:37.755525Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:10:37.755546Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:10:37.755553Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T14:10:37.830624Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T14:10:37.659Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 72 314 96 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "a7fa1b67-6f47-46b6-8c7c-9b9707cfbb29" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.133.0.28:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.30:58236 10.132.0.30:443 10.132.0.12:56422 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T14:10:38.169Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 102 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "b5d28125-7acb-4cbd-b7b3-774236d4ad63" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.30:40386 10.132.0.30:443 10.132.0.12:56436 - v1-completions-path 2026-10-30T14:10:41.352559Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:10:41.352598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:10:41.352614Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T14:10:41.352619Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T14:10:40.572Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 130 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "e99a0cdb-3f02-4831-94b6-94ba1c522557" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.30:40376 10.132.0.30:443 10.132.0.12:56442 - v1-completions-path [2026-10-30T14:10:41.276Z] "DELETE /v1/api-keys/2667f9a8-1d35-4524-830b-4094150ff190 HTTP/1.1" 200 - via_upstream - "-" 0 369 75 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "70fbcd42-f857-4a31-9538-a4cc5512715c" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.133.1.55:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.30:48444 10.132.0.30:443 10.132.0.12:56458 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T14:10:53.302696Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:11:03.938058Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:11:08.798036Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:11:12.457113Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T14:11:12.457159Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T14:11:12.457179Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T14:11:12.457186Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T14:11:12.379Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 67 309 77 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "da46e2b0-2910-48ed-9081-c18a3c289622" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.133.1.55:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.30:35994 10.132.0.30:443 10.132.0.12:46570 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T14:11:12.565Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 60 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "b7e1066d-fb2e-4cc5-9c64-60b2f245563e" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.30:40386 10.132.0.30:443 10.132.0.12:46584 - v1-completions-path 2026-10-30T14:11:25.917824Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:11:30.322863Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:11:33.297617Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T14:11:40.532Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 63 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "a21e83f2-d158-442a-bc54-271b481724ee" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.30:40386 10.132.0.30:443 10.132.0.12:40270 - v1-completions-path [2026-10-30T14:11:40.718Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 58 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "d8a41c61-e8ae-4a84-be86-9b18b685b215" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.30:443 10.132.0.12:40276 - v1-completions-path [2026-10-30T14:11:42.072Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 63 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "d2f18235-c3fe-4701-8419-fda1278e820e" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.30:40386 10.132.0.30:443 10.132.0.12:40290 - v1-completions-path [2026-10-30T14:11:42.984Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 61 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "5984c28a-6db1-4be2-9f2c-4f81cce5dec5" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.30:40376 10.132.0.30:443 10.132.0.12:40302 - v1-completions-path 2026-10-30T14:11:44.410754Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T14:11:44.410793Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T14:11:44.410810Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T14:11:44.410815Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T14:11:44.336Z] "DELETE /v1/api-keys/28fa988c-71eb-4f7f-a736-cb6a9bced3ea HTTP/1.1" 200 - via_upstream - "-" 0 363 73 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "dcd251b1-f9c6-425d-867c-a304d23c70f9" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.132.0.83:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.30:52352 10.132.0.30:443 10.132.0.12:40314 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T14:11:56.523314Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:12:06.784011Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:12:11.361160Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:12:15.226738Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:12:20.345139Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T14:12:36.030Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 74 - "52.71.36.72,10.132.0.12" "python-requests/2.34.2" "fb3cb42b-f76c-4984-90ab-45c75da13c74" "maas.apps.8221dcd364f53fb28015.prod.konflux-ocp-ci.dev" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.30:40386 10.132.0.30:443 10.132.0.12:49170 - v1-completions-path 2026-10-30T14:12:48.652115Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:12:58.909634Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:13:01.107701Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:13:13.698214Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:13:18.634739Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:13:35.445646Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:13:55.212198Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:14:05.638608Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:14:15.912852Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:14:26.292343Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:14:36.644515Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:14:47.054023Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:14:57.320185Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:15:07.706883Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:15:17.982465Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T14:15:28.325818Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8