2026-11-03T00:20:41.879545Z info FLAG: --concurrency="0" 2026-11-03T00:20:41.879574Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T00:20:41.879578Z info FLAG: --help="false" 2026-11-03T00:20:41.879580Z info FLAG: --log_as_json="false" 2026-11-03T00:20:41.879583Z info FLAG: --log_caller="" 2026-11-03T00:20:41.879585Z info FLAG: --log_output_level="default:info" 2026-11-03T00:20:41.879587Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T00:20:41.879591Z info FLAG: --log_target="[stdout]" 2026-11-03T00:20:41.879594Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T00:20:41.879598Z info FLAG: --outlierLogPath="" 2026-11-03T00:20:41.879600Z info FLAG: --profiling="true" 2026-11-03T00:20:41.879602Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T00:20:41.879604Z info FLAG: --proxyLogLevel="warning" 2026-11-03T00:20:41.879607Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T00:20:41.879609Z info FLAG: --stsPort="0" 2026-11-03T00:20:41.879611Z info FLAG: --templateFile="" 2026-11-03T00:20:41.879613Z info FLAG: --tokenManagerPlugin="" 2026-11-03T00:20:41.879615Z info FLAG: --vklog="0" 2026-11-03T00:20:41.879618Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T00:20:41.879794Z info Proxy role ips=[10.134.0.206] type=router id=maas-default-gateway-openshift-default-694db55c87-7zbtz.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T00:20:41.879836Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T00:20:41.881128Z info cpu limit detected as 2, setting concurrency 2026-11-03T00:20:41.881395Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T00:20:41.881408Z info JWT policy is third-party-jwt 2026-11-03T00:20:41.881411Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T00:20:41.883779Z info Starting default Istio SDS Server 2026-11-03T00:20:41.883817Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T00:20:41.883850Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T00:20:41.884449Z info Opening status port 15020 2026-11-03T00:20:41.885108Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T00:20:41.886320Z info sds Starting SDS grpc server 2026-11-03T00:20:41.886337Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T00:20:41.886772Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T00:20:41.887652Z info Starting proxy agent 2026-11-03T00:20:41.887685Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T00:20:41.952540Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T00:20:42.015007Z info cache generated new workload certificate resourceName=default latency=128.589338ms ttl=23h59m59.984996607s 2026-11-03T00:20:42.015050Z info cache Root cert has changed, start rotating root cert 2026-11-03T00:20:42.015097Z info cache returned workload trust anchor from cache ttl=23h59m59.984903943s 2026-11-03T00:20:42.124955Z info ads ADS: new connection for node:1 2026-11-03T00:20:42.125047Z info cache returned workload certificate from cache ttl=23h59m59.874955239s 2026-11-03T00:20:42.126726Z info ads ADS: new connection for node:2 2026-11-03T00:20:42.126808Z info cache returned workload trust anchor from cache ttl=23h59m59.873193877s 2026-11-03T00:20:42.212347Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:20:50.402273Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:20:51.363774Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T00:20:52.656394Z info Readiness succeeded in 10.780035764s 2026-11-03T00:20:52.656727Z info Envoy proxy is ready 2026-11-03T00:20:54.823151Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:21:00.756797Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-11-03T00:21:00.645Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 111 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "6758da3e-524c-4c46-a138-20e7b510e74b" "maas.apps.0f3a63f01c0b122488b1.prod.konflux-ocp-ci.dev" "10.133.0.35:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.206:43014 10.134.0.206:443 10.132.0.17:59774 - v1-catch-all-path 2026-11-03T00:21:08.692408Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:21:18.127290Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:21:28.683025Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:21:46.649928Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:21:49.007902Z info ads ADS: new connection for node:3 2026-11-03T00:21:49.008144Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T00:21:49.008186Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T00:21:56.174686Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:22:32.583543Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:22:56.499821Z info ads ADS: "" 3 terminated 2026-11-03T00:22:56.560666Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:23:01.374924Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:23:17.859031Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:25:36.395852Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:25:36.395887Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:25:36.395899Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:25:36.395904Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T00:25:36.646627Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:25:36.646662Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:25:36.646674Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:25:36.646680Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T00:25:36.318Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 77 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "c3750261-5f7d-49aa-852a-6106008cfb09" "maas.apps.0f3a63f01c0b122488b1.prod.konflux-ocp-ci.dev" "10.134.0.25:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.206:34100 10.134.0.206:443 10.132.0.17:59662 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T00:25:36.586Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 59 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "049fe0d5-bd62-4c0b-baa4-4b89f4768fdb" "maas.apps.0f3a63f01c0b122488b1.prod.konflux-ocp-ci.dev" "10.134.0.25:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.206:34106 10.134.0.206:443 10.132.0.17:59670 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T00:25:57.201628Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:26:28.512269Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:26:59.119260Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:27:14.581914Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:27:31.561632Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:27:44.098415Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:27:49.000817Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T00:27:54.742Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 79 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "2761c8b8-f56a-4c25-9668-941edb654b82" "maas.apps.0f3a63f01c0b122488b1.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.206:37534 10.134.0.206:443 10.132.0.17:58764 - v1-completions-path 2026-11-03T00:28:09.482738Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:28:56.861499Z info ads ADS: new connection for node:4 2026-11-03T00:28:56.865867Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-03T00:28:56.865932Z info cache read certificate from file resource=file-root:system 2026-11-03T00:29:00.782233Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:29:01.973427Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:29:04.766316Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:29:04.766356Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:29:04.766371Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T00:29:04.766376Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T00:29:04.642Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 73 315 124 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "72463fbb-8263-45da-98bc-d4bf4e44a1e9" "maas.apps.0f3a63f01c0b122488b1.prod.konflux-ocp-ci.dev" "10.134.0.25:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.206:47782 10.134.0.206:443 10.132.0.17:46318 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T00:29:21.118604Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:29:21.118638Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:29:21.118649Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:29:21.118653Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T00:29:21.118820Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T00:29:21.001Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1720 117 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "962e94f5-128d-48ad-bee8-fe6d8a57bfd2" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.206:46592 10.134.0.206:443 10.132.0.17:59580 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T00:29:51.385909Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:29:51.385940Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:29:51.385949Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T00:29:51.385954Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:29:51.386107Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T00:29:51.260Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1719 125 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "6e8e7d61-af86-482c-9a30-6bcb05afc11c" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.206:32958 10.134.0.206:443 10.132.0.17:34620 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T00:30:37.557797Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-03T00:30:37.667270Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:30:37.667307Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:30:37.667319Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T00:30:37.667323Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:30:37.667540Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T00:30:37.550Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1715 117 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "70e7ca8b-8959-419e-a0a2-9f95a4df6287" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.206:46592 10.134.0.206:443 10.132.0.17:50112 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T00:30:44.229951Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:30:44.229997Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:30:44.230015Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:30:44.230020Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T00:30:44.113Z] "DELETE /v1/api-keys/7997c32a-8a6f-4dad-98cd-6470c12cf13f HTTP/1.1" 200 - via_upstream - "-" 0 359 116 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "955cc638-9941-4093-a01c-683a8ba3b6fe" "maas.apps.0f3a63f01c0b122488b1.prod.konflux-ocp-ci.dev" "10.134.0.25:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.206:34306 10.134.0.206:443 10.132.0.17:56006 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T00:30:57.486221Z info ads ADS: "" 4 terminated 2026-11-03T00:30:57.529448Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:31:00.256594Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:31:02.016967Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T00:31:02.016998Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T00:31:02.017001Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T00:31:02.017005Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-11-03T00:31:01.942Z] "POST /llm/premium-simulated-simulated-premium/v1/completions HTTP/1.1" 403 - - "-" 74 77 74 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "bd0ae327-8726-4fb3-99c7-4b3317c57af5" "maas.apps.0f3a63f01c0b122488b1.prod.konflux-ocp-ci.dev" "-" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.206:443 10.132.0.17:47730 - v1-completions-path 2026-11-03T00:31:10.908472Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T00:31:14.638Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 67 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "cfa98d88-ff2f-46da-9427-7d02bca96e08" "maas.apps.0f3a63f01c0b122488b1.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.206:35482 10.134.0.206:443 10.132.0.17:52130 - v1-completions-path 2026-11-03T00:31:29.393099Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:31:34.043249Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:31:40.804373Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:31:40.804412Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:31:40.804427Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T00:31:40.804432Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T00:31:40.742Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 170 61 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "8c574dbf-eada-488a-be8e-8aa965560dec" "maas.apps.0f3a63f01c0b122488b1.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.206:37534 10.134.0.206:443 10.132.0.17:39508 - v1-catch-all-path 2026-11-03T00:31:58.142102Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:32:11.351690Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:32:19.232796Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-03T00:32:19.226Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 63 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "85b4f74c-85cf-4e3e-9120-4f9cba26ea90" "maas.apps.0f3a63f01c0b122488b1.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.206:37534 10.134.0.206:443 10.132.0.17:48112 - v1-completions-path [2026-11-03T00:32:20.321Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 55 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "02157443-7009-4311-9214-cce717c93b24" "maas.apps.0f3a63f01c0b122488b1.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.206:443 10.132.0.17:48136 - v1-completions-path [2026-11-03T00:32:20.833Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 60 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "ba8efdad-cc94-4703-a1eb-c30bd859cb59" "maas.apps.0f3a63f01c0b122488b1.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.206:37534 10.134.0.206:443 10.132.0.17:48152 - v1-completions-path [2026-11-03T00:32:22.087Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 58 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "c1a61c93-cbcc-43b9-a350-171149b7e94e" "maas.apps.0f3a63f01c0b122488b1.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.206:37534 10.134.0.206:443 10.132.0.17:48156 - v1-completions-path [2026-11-03T00:32:23.155Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 59 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "a37bceb4-83f5-9a13-a8e1-15571571b595" "maas.apps.0f3a63f01c0b122488b1.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.206:37534 10.134.0.206:443 10.132.0.17:48164 - v1-completions-path 2026-11-03T00:32:42.211993Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:32:52.921391Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:32:59.422683Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T00:33:13.955Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 60 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "bfbf89e2-4ded-4fff-9229-860b599ad963" "maas.apps.0f3a63f01c0b122488b1.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.206:37534 10.134.0.206:443 10.132.0.17:42540 - v1-completions-path 2026-11-03T00:33:27.763377Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:33:32.329499Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:33:32.712059Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:33:32.712099Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:33:32.712117Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T00:33:32.712122Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T00:33:32.422Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 73 315 290 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "2fd00f2d-dd9f-4af0-90d5-f5f7bd072f69" "maas.apps.0f3a63f01c0b122488b1.prod.konflux-ocp-ci.dev" "10.134.0.25:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.206:56142 10.134.0.206:443 10.132.0.17:55826 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T00:33:45.661946Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:33:52.264368Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:34:06.713133Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:34:11.431065Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:34:24.995128Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:34:38.369511Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:34:44.334361Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:35:02.775517Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:35:08.398597Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:35:10.209956Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:35:10.209995Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:35:10.210011Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T00:35:10.210016Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T00:35:10.098Z] "DELETE /v1/api-keys/26d44b0f-256a-4245-bb11-316ec70100c0 HTTP/1.1" 404 - via_upstream - "-" 0 29 111 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "15b58b8e-a21f-463e-a018-2e1bd7679644" "maas.apps.0f3a63f01c0b122488b1.prod.konflux-ocp-ci.dev" "10.134.0.25:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.206:40490 10.134.0.206:443 10.132.0.17:33200 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T00:35:19.865129Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:35:46.368623Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:35:57.052608Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:36:00.210755Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:36:00.210796Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T00:36:00.210814Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T00:36:00.210819Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T00:36:00.095Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 115 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "9b32e432-017e-41db-bd00-2a1f2948e1ae" "maas.apps.0f3a63f01c0b122488b1.prod.konflux-ocp-ci.dev" "10.134.0.25:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.206:40490 10.134.0.206:443 10.132.0.17:52094 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T00:36:15.706969Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:36:23.432076Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:39:20.101052Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:40:48.515959Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:41:05.900780Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:41:10.680747Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:41:24.033675Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:41:39.813435Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:41:44.742101Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T00:41:48.002Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 75 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "0d604d09-21c1-4735-9af7-a33d0c8b8b6c" "maas.apps.0f3a63f01c0b122488b1.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.206:37534 10.134.0.206:443 10.132.0.17:52522 - v1-completions-path 2026-11-03T00:42:04.666468Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:42:15.463277Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:42:21.810062Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:42:51.005126Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:42:57.422576Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:43:03.538308Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T00:43:20.640Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 74 - "44.212.242.249,10.132.0.17" "python-requests/2.34.2" "6e613183-7d4a-4fa2-adde-a6398d114d65" "maas.apps.0f3a63f01c0b122488b1.prod.konflux-ocp-ci.dev" "10.133.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.206:35482 10.134.0.206:443 10.132.0.17:60126 - v1-completions-path 2026-11-03T00:43:35.681691Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:43:46.272762Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:43:52.796759Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:43:58.763710Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:44:11.053836Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:44:34.497729Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:44:45.112345Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:44:50.004115Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:45:08.362438Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:45:15.354879Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:45:24.765187Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:45:45.101524Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:45:57.798934Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:46:17.930797Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:46:28.972154Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:46:39.907069Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:46:50.559659Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:47:01.371190Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:47:12.153094Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:47:22.895793Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T00:47:33.537344Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8