2026-10-30T14:01:37.562357Z info FLAG: --concurrency="0" 2026-10-30T14:01:37.562378Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T14:01:37.562382Z info FLAG: --help="false" 2026-10-30T14:01:37.562385Z info FLAG: --log_as_json="false" 2026-10-30T14:01:37.562387Z info FLAG: --log_caller="" 2026-10-30T14:01:37.562408Z info FLAG: --log_output_level="default:info" 2026-10-30T14:01:37.562412Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T14:01:37.562419Z info FLAG: --log_target="[stdout]" 2026-10-30T14:01:37.562423Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T14:01:37.562426Z info FLAG: --outlierLogPath="" 2026-10-30T14:01:37.562430Z info FLAG: --profiling="true" 2026-10-30T14:01:37.562433Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T14:01:37.562435Z info FLAG: --proxyLogLevel="warning" 2026-10-30T14:01:37.562437Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T14:01:37.562440Z info FLAG: --stsPort="0" 2026-10-30T14:01:37.562441Z info FLAG: --templateFile="" 2026-10-30T14:01:37.562444Z info FLAG: --tokenManagerPlugin="" 2026-10-30T14:01:37.562446Z info FLAG: --vklog="0" 2026-10-30T14:01:37.562449Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T14:01:37.562602Z info Proxy role ips=[10.133.0.239] type=router id=maas-default-gateway-openshift-default-68cb9b4b95-rvmhw.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T14:01:37.562664Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T14:01:37.563735Z info cpu limit detected as 2, setting concurrency 2026-10-30T14:01:37.563940Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T14:01:37.563952Z info JWT policy is third-party-jwt 2026-10-30T14:01:37.563955Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T14:01:37.566988Z info Starting default Istio SDS Server 2026-10-30T14:01:37.567026Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T14:01:37.567056Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T14:01:37.567267Z info Opening status port 15020 2026-10-30T14:01:37.567897Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T14:01:37.569121Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T14:01:37.569918Z info Starting proxy agent 2026-10-30T14:01:37.569949Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T14:01:37.577683Z info sds Starting SDS grpc server 2026-10-30T14:01:37.577758Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T14:01:37.644372Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T14:01:37.776318Z info ads ADS: new connection for node:1 2026-10-30T14:01:37.777782Z info ads ADS: new connection for node:2 2026-10-30T14:01:37.806895Z info cache generated new workload certificate resourceName=default latency=238.779837ms ttl=23h59m59.193110793s 2026-10-30T14:01:37.806942Z info cache Root cert has changed, start rotating root cert 2026-10-30T14:01:37.806994Z info cache returned workload trust anchor from cache ttl=23h59m59.19300733s 2026-10-30T14:01:37.807014Z info cache returned workload certificate from cache ttl=23h59m59.192987435s 2026-10-30T14:01:37.807098Z info cache returned workload trust anchor from cache ttl=23h59m59.192904853s 2026-10-30T14:01:37.807522Z info cache returned workload trust anchor from cache ttl=23h59m59.192479921s 2026-10-30T14:01:37.893995Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:01:46.249458Z info Readiness succeeded in 8.689555543s 2026-10-30T14:01:46.250224Z info Envoy proxy is ready 2026-10-30T14:02:09.686760Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:02:13.705825Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:02:13.705864Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:02:13.705878Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:02:13.705883Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T14:02:13.618Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 61 304 87 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "92797ea8-92fc-44b1-927d-2534183d339e" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "10.133.0.27:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.239:41798 10.133.0.239:443 10.132.0.13:54580 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T14:02:26.561948Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:02:29.288290Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:02:32.555629Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-10-30T14:02:39.836756Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:02:45.876965Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-10-30T14:02:45.796Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 80 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "06bc0453-4245-4e1a-94d2-973858f62116" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.239:54108 10.133.0.239:443 10.132.0.13:55196 - v1-catch-all-path 2026-10-30T14:02:58.071893Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:03:00.440935Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:03:15.331299Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:03:26.391639Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:03:28.439257Z info ads ADS: new connection for node:3 2026-10-30T14:03:28.439470Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T14:03:28.439503Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T14:03:30.339878Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:04:11.098888Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-30T14:04:14.853Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 334 87 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "d20ac119-a242-479f-848f-96c7c5e137b8" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "10.132.1.3:8000" outbound|54321||e2e-prio-pool-6a03d47f-inference-pool-ip-d0654a22.llm.svc.cluster.local 10.133.0.239:37540 10.133.0.239:443 10.132.0.13:59784 - v1-model-routing 2026-10-30T14:04:30.260806Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:04:30.287006Z info ads ADS: "" 3 terminated 2026-10-30T14:04:33.660646Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:04:41.649990Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:06:15.438031Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:06:33.606641Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:06:48.392969Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:07:00.511298Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:07:04.138705Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:07:14.381990Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:07:18.915557Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-30T14:07:24.540Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 70 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "4c9646c1-f7a4-4d36-a985-56af4d0f9976" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "10.132.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.239:32794 10.133.0.239:443 10.132.0.13:49716 - v1-completions-path 2026-10-30T14:07:37.022606Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:08:23.514795Z info ads ADS: new connection for node:4 2026-10-30T14:08:23.522386Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-10-30T14:08:23.522529Z info cache read certificate from file resource=file-root:system 2026-10-30T14:08:24.414963Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:08:26.211375Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:08:31.239291Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"136","StreamId":"10779496067947146639"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=24 [2026-10-30T14:08:31.177Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 62 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "7d8995d8-4350-4505-ba3a-998322a6d00a" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.133.0.239:443 10.132.0.13:37296 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-10-30T14:08:39.920169Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T14:08:39.920205Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T14:08:39.920216Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T14:08:39.920221Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T14:08:39.920369Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T14:08:39.810Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1713 109 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "4f9421dd-439c-48c2-af36-947786f07c1c" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.239:41648 10.133.0.239:443 10.132.0.13:55100 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T14:08:44.170576Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T14:08:44.170613Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T14:08:44.170627Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T14:08:44.170633Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T14:08:44.170826Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T14:08:44.079Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1721 91 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "85768d30-b884-468d-9b66-96ae7ee7a431" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.133.0.239:41648 10.133.0.239:443 10.132.0.13:55116 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T14:09:50.900974Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:09:50.983103Z info ads ADS: "" 4 terminated 2026-10-30T14:09:51.167938Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-10-30T14:09:51.238960Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:09:51.238997Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:09:51.239012Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T14:09:51.239017Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T14:09:51.159Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 74 317 79 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "4d1009ba-8656-45d3-adab-05c120c525e5" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "10.133.0.27:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.239:37808 10.133.0.239:443 10.132.0.13:36814 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T14:09:52.493800Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:09:59.177230Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:10:16.386217Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:10:21.000731Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:10:36.479202Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:10:52.406144Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-30T14:10:57.747Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 289 62 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "6f9576af-5e0b-4c22-bc44-15fcead5d84b" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "10.132.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.239:32794 10.133.0.239:443 10.132.0.13:41204 - v1-completions-path [2026-10-30T14:10:57.835Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 57 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "448056b5-4772-46f2-ba13-6e93eab80747" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "10.132.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.239:32794 10.133.0.239:443 10.132.0.13:41208 - v1-completions-path 2026-10-30T14:10:58.795912Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-10-30T14:10:58.397Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 58 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "7ef8fa44-4a24-4253-9990-09535007f4a8" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "10.132.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.239:32794 10.133.0.239:443 10.132.0.13:41218 - v1-completions-path [2026-10-30T14:10:58.741Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 62 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "2c400146-85e4-43b6-b87d-6528dcd35a1f" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "10.132.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.239:60902 10.133.0.239:443 10.132.0.13:41232 - v1-completions-path [2026-10-30T14:10:59.063Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 54 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "1932b480-e074-4a3d-a2df-148f4510d093" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "10.132.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.239:60902 10.133.0.239:443 10.132.0.13:41244 - v1-completions-path [2026-10-30T14:10:59.868Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 56 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "517581ac-a56e-4dba-9085-c132a4dbb077" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "10.132.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.239:60902 10.133.0.239:443 10.132.0.13:41270 - v1-completions-path 2026-10-30T14:11:15.999427Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:11:26.337020Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:11:31.046128Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:11:47.911930Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:11:50.437821Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:12:09.620383Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-30T14:12:11.411Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 281 82 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "75f01cd2-9d8c-4ed3-8963-2ba2cdc1822b" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "10.132.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.239:33126 10.133.0.239:443 10.132.0.13:36646 - v1-completions-path 2026-10-30T14:12:14.277807Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:12:29.798241Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:12:32.141420Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:12:50.024505Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:13:00.811138Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:13:03.839297Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:13:25.089221Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T14:13:25.089259Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T14:13:25.089274Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T14:13:25.089279Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T14:13:25.016Z] "DELETE /v1/api-keys/48e18c4e-632c-4188-aa67-bd33809674a7 HTTP/1.1" 200 - via_upstream - "-" 0 375 72 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "90f4fdfd-8e02-431f-922b-e2c7b360f21b" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "10.133.0.27:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.239:50894 10.133.0.239:443 10.132.0.13:49072 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T14:13:31.843590Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-30T14:13:37.648Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 298 62 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "c4b280ef-8f93-457d-a6f1-a6bc1ec02a9b" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "10.132.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.239:32794 10.133.0.239:443 10.132.0.13:37972 - v1-completions-path 2026-10-30T14:13:49.955780Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:14:00.195145Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:17:36.181521Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:17:51.388301Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:17:53.655164Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:18:12.778965Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:18:25.012018Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:18:27.939404Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:18:43.635648Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:18:54.487579Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:18:59.479948Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:19:03.234187Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:19:03.234223Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:19:03.234240Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T14:19:03.234244Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T14:19:03.156Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 67 310 77 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "745fad01-e0ef-4343-95b1-72463b7fd947" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "10.133.0.249:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.239:34448 10.133.0.239:443 10.132.0.13:51764 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T14:19:03.803Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 58 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "e8b260b4-5c04-4512-973c-cc4873678950" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "10.132.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.239:32794 10.133.0.239:443 10.132.0.13:51768 - v1-completions-path 2026-10-30T14:19:04.438279Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T14:19:04.438314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T14:19:04.438328Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T14:19:04.438333Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T14:19:04.561988Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:19:04.562030Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T14:19:04.562051Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T14:19:04.562057Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-30T14:19:04.169Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 51 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "39c4a2b0-14ec-474c-86df-dc1228b83940" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.239:443 10.132.0.13:51784 - v1-completions-path [2026-10-30T14:19:04.325Z] "DELETE /v1/api-keys/f2987f38-75de-4827-bb41-67b9bd3300a5 HTTP/1.1" 200 - via_upstream - "-" 0 365 112 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "2ec5e1f2-192e-4030-86b5-4a022f0b45ff" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "10.132.1.44:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.239:59540 10.133.0.239:443 10.132.0.13:51794 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T14:19:04.455Z] "DELETE /v1/api-keys/f015a417-a239-4d51-a92d-1ff7f0b5cbe2 HTTP/1.1" 200 - via_upstream - "-" 0 365 106 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "59e7429e-082c-4ea7-8c0f-79ac8ab3ce30" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "10.132.1.44:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.239:59546 10.133.0.239:443 10.132.0.13:51800 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T14:19:16.542618Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:19:21.073768Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:19:24.663693Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-30T14:19:31.599Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 57 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "73e3d4b8-0ba8-4613-8e48-f4ec3b7e5446" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.239:443 10.132.0.13:33076 - v1-completions-path [2026-10-30T14:19:32.042Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 56 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "02fef563-f315-4238-a061-f975f5387cda" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "10.132.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.239:32794 10.133.0.239:443 10.132.0.13:33080 - v1-completions-path 2026-10-30T14:19:47.383524Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:19:57.745170Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:20:02.351363Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:20:05.964135Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:20:10.658217Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-30T14:20:16.511Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 62 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "d4551b31-a991-4d43-997a-63eb51a22d55" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "10.132.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.239:60902 10.133.0.239:443 10.132.0.13:59496 - v1-completions-path 2026-10-30T14:20:29.093742Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:20:39.414070Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:20:41.690680Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:20:51.366015Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:20:55.006352Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:21:13.349034Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:21:33.385843Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:21:43.809729Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:21:54.072635Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:22:04.387523Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:22:14.914964Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:22:25.225850Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:22:35.681643Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:22:46.135542Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:22:56.542604Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:23:06.818663Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:23:16.107242Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T14:23:16.107268Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T14:23:16.107271Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T14:23:16.107275Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-10-30T14:23:16.080Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 401 - - "-" 66 88 26 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "ed8850b9-e15c-4e6d-ab14-01c6e8f28536" "maas.apps.0d483a2e2de579d84691.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.239:443 10.132.0.13:58056 - v1-completions-path