2026-10-30T07:34:32.965520Z info FLAG: --concurrency="0" 2026-10-30T07:34:32.965537Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T07:34:32.965540Z info FLAG: --help="false" 2026-10-30T07:34:32.965543Z info FLAG: --log_as_json="false" 2026-10-30T07:34:32.965545Z info FLAG: --log_caller="" 2026-10-30T07:34:32.965547Z info FLAG: --log_output_level="default:info" 2026-10-30T07:34:32.965549Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T07:34:32.965553Z info FLAG: --log_target="[stdout]" 2026-10-30T07:34:32.965556Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T07:34:32.965557Z info FLAG: --outlierLogPath="" 2026-10-30T07:34:32.965559Z info FLAG: --profiling="true" 2026-10-30T07:34:32.965561Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T07:34:32.965564Z info FLAG: --proxyLogLevel="warning" 2026-10-30T07:34:32.965565Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T07:34:32.965567Z info FLAG: --stsPort="0" 2026-10-30T07:34:32.965569Z info FLAG: --templateFile="" 2026-10-30T07:34:32.965571Z info FLAG: --tokenManagerPlugin="" 2026-10-30T07:34:32.965573Z info FLAG: --vklog="0" 2026-10-30T07:34:32.965576Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T07:34:32.965723Z info Proxy role ips=[10.132.1.164] type=router id=maas-default-gateway-openshift-default-85544d4cd7-4j2fq.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T07:34:32.965774Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T07:34:32.966886Z info cpu limit detected as 2, setting concurrency 2026-10-30T07:34:32.967534Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T07:34:32.967558Z info JWT policy is third-party-jwt 2026-10-30T07:34:32.967564Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T07:34:32.970044Z info Starting default Istio SDS Server 2026-10-30T07:34:32.970080Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T07:34:32.970107Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T07:34:32.970336Z info Opening status port 15020 2026-10-30T07:34:32.971951Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T07:34:32.972458Z info sds Starting SDS grpc server 2026-10-30T07:34:32.972477Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T07:34:32.973693Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T07:34:32.974905Z info Starting proxy agent 2026-10-30T07:34:32.974953Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T07:34:33.035920Z info cache generated new workload certificate resourceName=default latency=63.290224ms ttl=23h59m59.964085601s 2026-10-30T07:34:33.035958Z info cache Root cert has changed, start rotating root cert 2026-10-30T07:34:33.035998Z info cache returned workload trust anchor from cache ttl=23h59m59.964003278s 2026-10-30T07:34:33.046785Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T07:34:33.190859Z info ads ADS: new connection for node:1 2026-10-30T07:34:33.190929Z info cache returned workload certificate from cache ttl=23h59m59.809072452s 2026-10-30T07:34:33.191810Z info ads ADS: new connection for node:2 2026-10-30T07:34:33.191899Z info cache returned workload trust anchor from cache ttl=23h59m59.808103475s 2026-10-30T07:34:33.276412Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:34:41.313510Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:34:42.237674Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-10-30T07:34:42.571410Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-10-30T07:34:43.637998Z info Readiness succeeded in 10.675524995s 2026-10-30T07:34:43.638371Z info Readiness succeeded in 10.675907415s 2026-10-30T07:34:43.638483Z info Envoy proxy is ready 2026-10-30T07:35:08.372202Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:35:26.499095Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:35:28.762456Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:35:39.675195Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:35:57.712336Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:36:00.048279Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:36:14.832600Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:36:25.843485Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:36:27.717798Z info ads ADS: new connection for node:3 2026-10-30T07:36:27.717990Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T07:36:27.718018Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-10-30T07:36:29.897588Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:37:10.412916Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:37:29.417351Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:37:29.464540Z info ads ADS: "" 3 terminated with stream closed 2026-10-30T07:37:33.030900Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:37:40.637576Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:39:16.046751Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:39:32.482747Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:39:47.831111Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:40:01.942248Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:40:06.763030Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:40:19.322445Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:40:22.916765Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:40:31.072062Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:40:31.072107Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:40:31.072125Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T07:40:31.072130Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T07:40:30.999Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 524 72 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "39d7e238-fc76-4751-987e-61d44ce8fbcc" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.164:33646 10.132.1.164:443 10.132.0.21:37696 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T07:40:43.062247Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T07:41:15.061Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 281 88 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "afdf52b3-472f-43f4-9b65-5dc6e8fa9356" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.1.164:51260 10.132.1.164:443 10.132.0.21:54808 - v1-completions-path 2026-10-30T07:41:30.323616Z info ads ADS: new connection for node:4 2026-10-30T07:41:30.336120Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-10-30T07:41:30.336277Z info cache read certificate from file resource=file-root:system 2026-10-30T07:41:31.408805Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:41:32.987392Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:42:22.447009Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:42:22.447042Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:42:22.447051Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:42:22.447055Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T07:42:22.447273Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-30T07:42:22.326Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1719 120 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "abfb3199-c8a5-4e80-89d3-8ab6b0b64d9b" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.1.164:52280 10.132.1.164:443 10.132.0.21:55482 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T07:42:33.189643Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T07:42:33.189680Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T07:42:33.189691Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T07:42:33.189696Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-30T07:42:33.189878Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-10-30T07:42:33.079Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1728 110 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "0314ddd5-0b13-41ad-be7b-da00df569f8b" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.1.164:50596 10.132.1.164:443 10.132.0.21:38504 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-10-30T07:42:47.755585Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:42:47.755623Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:42:47.755639Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:42:47.755644Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T07:42:47.673Z] "DELETE /v1/api-keys/d0969a34-b663-4d14-9422-a5dd06bce37e HTTP/1.1" 200 - via_upstream - "-" 0 359 81 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "9a23a95d-ffe5-4fcd-9a57-2b9bf59bd341" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.164:34274 10.132.1.164:443 10.132.0.21:48900 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T07:42:59.341277Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:42:59.470095Z info ads ADS: "" 4 terminated with stream closed 2026-10-30T07:43:00.893418Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:43:07.776941Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:43:24.959690Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:43:29.457726Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:43:32.824494Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:43:32.824534Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:43:32.824548Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T07:43:32.824553Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T07:43:32.749Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 336 74 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "db550bdf-6f9b-41d6-9f63-02b73816e710" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.164:34274 10.132.1.164:443 10.132.0.21:48012 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T07:43:33.237052Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:43:33.237091Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:43:33.237105Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:43:33.237111Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T07:43:33.167Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 170 69 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "bb5f62aa-df7c-4fd9-81d0-e72bc6393d53" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.164:34754 10.132.1.164:443 10.132.0.21:48016 - v1-catch-all-path 2026-10-30T07:43:45.197058Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:44:01.038845Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:44:06.559161Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-10-30T07:44:06.503Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 62 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "de505f65-bd3b-4492-8449-837f5283f053" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.164:34754 10.132.1.164:443 10.132.0.21:43672 - v1-completions-path [2026-10-30T07:44:07.167Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 62 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "debcfd14-058c-4ddd-8e04-3535cbce37b7" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.164:34754 10.132.1.164:443 10.132.0.21:43684 - v1-completions-path 2026-10-30T07:44:23.588159Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:44:33.896382Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:44:38.743151Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T07:44:45.451Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 299 61 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "58d98ed4-9945-4acf-aa46-6a8284258002" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.164:34754 10.132.1.164:443 10.132.0.21:44546 - v1-completions-path 2026-10-30T07:44:58.747526Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:45:01.029120Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:45:20.553852Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:45:22.907695Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:45:41.541381Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:45:43.858138Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:46:01.739348Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:46:12.418877Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:46:15.302861Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:46:34.602215Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:46:37.425645Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:46:50.031486Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T07:46:55.802Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 62 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "821b7aa3-ff75-4fdd-840b-2ac5720985e7" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.164:34754 10.132.1.164:443 10.132.0.21:47480 - v1-completions-path 2026-10-30T07:47:08.001609Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:47:18.331430Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:47:25.691429Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:47:25.691467Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:47:25.691485Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T07:47:25.691491Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T07:47:25.613Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 77 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "2667515b-d7ed-4b1d-9ea6-fc571c396c75" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.164:52298 10.132.1.164:443 10.132.0.21:36122 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T07:47:36.244Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 38 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "d2c9d22c-1b09-4d48-8f6c-abecd1321333" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.1.164:443 10.132.0.21:43984 - v1-completions-path 2026-10-30T07:47:38.419750Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:47:41.801108Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T07:49:22.700Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 46 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "38c5b58b-216a-4739-9fc4-569c7556a8ec" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.1.164:443 10.132.0.21:39078 - v1-completions-path 2026-10-30T07:49:24.897092Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:49:28.398153Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:50:39.747863Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:50:39.747904Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:50:39.747922Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:50:39.747927Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T07:50:39.670Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 77 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "00e9cc39-3663-4223-b1c3-621272917c98" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.30:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.164:34298 10.132.1.164:443 10.132.0.21:37780 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T07:51:01.751148Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:51:17.260004Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:51:19.754622Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:51:38.611067Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:51:50.606718Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:51:53.564599Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:52:09.432805Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:52:20.010656Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:52:24.656472Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:52:29.738605Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:52:29.738647Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:52:29.738667Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:52:29.738673Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T07:52:29.315Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 65 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "f035e69f-bcbb-43e4-adec-d2e9f53f642c" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.164:45162 10.132.1.164:443 10.132.0.21:36538 - v1-completions-path [2026-10-30T07:52:29.665Z] "DELETE /v1/api-keys/407392ff-9b26-4636-931b-eaca6d1c3654 HTTP/1.1" 200 - via_upstream - "-" 0 365 73 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "a7ae76ee-f603-494c-996a-bad83c6ff7ee" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.133.1.61:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.164:34102 10.132.1.164:443 10.132.0.21:36550 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T07:52:41.886192Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:52:46.477325Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:52:49.560530Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:52:58.325772Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-10-30T07:52:58.266Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 64 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "7da29d47-3bca-4e52-9619-b59facff8df1" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.164:45162 10.132.1.164:443 10.132.0.21:54454 - v1-completions-path 2026-10-30T07:53:00.512569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:53:00.512609Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:53:00.512627Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:53:00.512632Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T07:53:00.439Z] "DELETE /v1/api-keys/64566e9b-81a0-4549-8615-9d93f0af6a3e HTTP/1.1" 200 - via_upstream - "-" 0 365 73 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "69cccaf7-1e1b-4c62-87a3-f681d46b961c" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.222:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.164:38682 10.132.1.164:443 10.132.0.21:50722 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T07:53:12.521490Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:53:22.911259Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:53:27.465886Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:53:31.156966Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T07:53:33.863Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 70 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "460a1b5f-69bb-4ec7-a8ec-105dabab24d2" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.164:34754 10.132.1.164:443 10.132.0.21:55420 - v1-completions-path 2026-10-30T07:53:36.206658Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T07:53:41.550Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 65 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "d344b25a-8ffe-4475-b221-9dba2236424a" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.1.164:34754 10.132.1.164:443 10.132.0.21:56614 - v1-completions-path 2026-10-30T07:53:42.328219Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:53:42.328287Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T07:53:42.328304Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T07:53:42.328308Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T07:53:42.198Z] "DELETE /v1/api-keys/2ea9f8e2-2ce1-4ef2-a3a5-13ff5bca6d41 HTTP/1.1" 200 - via_upstream - "-" 0 367 130 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "bd2e08ce-d0aa-4439-aeeb-f43e9bed73dd" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "10.133.1.61:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.1.164:45050 10.132.1.164:443 10.132.0.21:56628 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T07:53:54.331817Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:54:04.715393Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:54:07.098348Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:54:18.456781Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:54:22.120556Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:54:39.794069Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:54:42.164305Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:55:00.450852Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:55:10.948656Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:55:21.086298Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:55:31.465790Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:55:41.872191Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:55:52.348060Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:56:02.495542Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:56:12.958658Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:56:23.180849Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:56:33.508348Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T07:56:36.257350Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T07:56:36.257374Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T07:56:36.257378Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-30T07:56:36.257382Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-10-30T07:56:36.233Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 401 - - "-" 66 88 23 - "44.212.242.249,10.132.0.21" "python-requests/2.34.2" "c70b287d-ff0c-408d-84b8-313ccff2306b" "maas.apps.66975bce54657ce23a19.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.1.164:443 10.132.0.21:47146 - v1-completions-path