2026-11-02T22:07:23.114358Z info FLAG: --concurrency="0" 2026-11-02T22:07:23.114377Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-02T22:07:23.114381Z info FLAG: --help="false" 2026-11-02T22:07:23.114383Z info FLAG: --log_as_json="false" 2026-11-02T22:07:23.114385Z info FLAG: --log_caller="" 2026-11-02T22:07:23.114387Z info FLAG: --log_output_level="default:info" 2026-11-02T22:07:23.114390Z info FLAG: --log_stacktrace_level="default:none" 2026-11-02T22:07:23.114394Z info FLAG: --log_target="[stdout]" 2026-11-02T22:07:23.114396Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-02T22:07:23.114398Z info FLAG: --outlierLogPath="" 2026-11-02T22:07:23.114400Z info FLAG: --profiling="true" 2026-11-02T22:07:23.114402Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-02T22:07:23.114404Z info FLAG: --proxyLogLevel="warning" 2026-11-02T22:07:23.114406Z info FLAG: --serviceCluster="istio-proxy" 2026-11-02T22:07:23.114408Z info FLAG: --stsPort="0" 2026-11-02T22:07:23.114410Z info FLAG: --templateFile="" 2026-11-02T22:07:23.114412Z info FLAG: --tokenManagerPlugin="" 2026-11-02T22:07:23.114414Z info FLAG: --vklog="0" 2026-11-02T22:07:23.114417Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-02T22:07:23.114634Z info Proxy role ips=[10.134.0.33] type=router id=maas-default-gateway-openshift-default-696f47c6c-skqp6.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-02T22:07:23.114705Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-02T22:07:23.115824Z info cpu limit detected as 2, setting concurrency 2026-11-02T22:07:23.116060Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-02T22:07:23.116072Z info JWT policy is third-party-jwt 2026-11-02T22:07:23.116074Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-02T22:07:23.117619Z info Starting default Istio SDS Server 2026-11-02T22:07:23.117641Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-02T22:07:23.117657Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-02T22:07:23.117956Z info Opening status port 15020 2026-11-02T22:07:23.118900Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-02T22:07:23.119517Z info sds Starting SDS grpc server 2026-11-02T22:07:23.119548Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-02T22:07:23.120322Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-02T22:07:23.121221Z info Starting proxy agent 2026-11-02T22:07:23.121253Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-02T22:07:23.147341Z info cache generated new workload certificate resourceName=default latency=27.707335ms ttl=23h59m59.852661041s 2026-11-02T22:07:23.147370Z info cache Root cert has changed, start rotating root cert 2026-11-02T22:07:23.147403Z info cache returned workload trust anchor from cache ttl=23h59m59.85259784s 2026-11-02T22:07:23.183933Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-02T22:07:23.280453Z info ads ADS: new connection for node:1 2026-11-02T22:07:23.280520Z info cache returned workload certificate from cache ttl=23h59m59.719481659s 2026-11-02T22:07:23.280723Z info ads ADS: new connection for node:2 2026-11-02T22:07:23.280788Z info cache returned workload trust anchor from cache ttl=23h59m59.719213368s 2026-11-02T22:07:23.357121Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:07:31.368126Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:07:32.482822Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-02T22:07:33.543012Z info Readiness succeeded in 10.431052729s 2026-11-02T22:07:33.543497Z info Envoy proxy is ready 2026-11-02T22:07:36.449653Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:07:37.078078Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T22:08:12.852Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 63 - "44.212.242.249,10.133.0.11" "curl/7.76.1" "bef8cc7a-585c-4383-868c-b21025d008d3" "maas.apps.1d4172f35d0cbb343988.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.33:443 10.133.0.11:41912 - v1-model-routing 2026-11-02T22:08:13.454253Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-02T22:08:13.454281Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-02T22:08:13.454284Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-02T22:08:13.454288Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-11-02T22:08:12.975Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 50 - "44.212.242.249,10.133.0.11" "curl/7.76.1" "f5a50b44-b6cd-456a-ac8f-0734aae348ad" "maas.apps.1d4172f35d0cbb343988.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.33:443 10.133.0.11:41918 - v1-model-routing [2026-11-02T22:08:13.434Z] "POST /v1/chat/completions HTTP/1.1" 401 - - "-" 122 88 20 - "44.212.242.249,10.133.0.11" "curl/7.76.1" "8074117f-3ae9-4e7f-90a5-50bba875d346" "maas.apps.1d4172f35d0cbb343988.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.33:443 10.133.0.11:41928 - v1-model-routing 2026-11-02T22:08:13.601202Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:08:13.601236Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:08:13.601251Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:08:13.601256Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-02T22:08:13.533Z] "DELETE /v1/api-keys/e4d27bc8-bec5-423f-8b52-d54c83e1c255 HTTP/1.1" 200 - via_upstream - "-" 0 365 67 - "44.212.242.249,10.133.0.11" "curl/7.76.1" "32a335f2-ede9-4cce-95ea-9eaba920c998" "maas.apps.1d4172f35d0cbb343988.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.33:36094 10.134.0.33:443 10.133.0.11:41936 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T22:10:19.392Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 83 0 63 - "44.212.242.249,10.133.0.11" "python-requests/2.34.2" "8e09bde3-3644-4817-9650-dc107a155711" "maas.apps.1d4172f35d0cbb343988.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.134.0.33:443 10.133.0.11:43126 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T22:10:21.016484Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:10:21.016523Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:10:21.016544Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:10:21.016550Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T22:10:20.919Z] "DELETE /v1/api-keys/962b9293-e32a-4f2e-b9d9-0e6d5dfb6d60 HTTP/1.1" 200 - via_upstream - "-" 0 321 96 - "44.212.242.249,10.133.0.11" "python-requests/2.34.2" "3e6339a4-fd5c-4b1d-bcfc-9e4edb35d499" "maas.apps.1d4172f35d0cbb343988.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.33:44352 10.134.0.33:443 10.133.0.11:46010 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T22:10:21.670992Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:10:21.671040Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:10:21.671060Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:10:21.671066Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T22:10:22.108142Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:10:22.108176Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:10:22.108190Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:10:22.108196Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T22:10:21.570Z] "DELETE /v1/api-keys/81f2a9b4-40f9-44aa-a14a-7bacb8d8b5ef HTTP/1.1" 200 - via_upstream - "-" 0 321 100 - "44.212.242.249,10.133.0.11" "python-requests/2.34.2" "87fb4a37-b185-4b3e-9297-be74e9ba259b" "maas.apps.1d4172f35d0cbb343988.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.33:33180 10.134.0.33:443 10.133.0.11:46028 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T22:10:22.015Z] "DELETE /v1/api-keys/6fdf6dc0-21d3-40a7-b885-27339c1990c4 HTTP/1.1" 200 - via_upstream - "-" 0 321 92 - "44.212.242.249,10.133.0.11" "python-requests/2.34.2" "3ca80317-f073-4bd4-a5a5-7da8f375769f" "maas.apps.1d4172f35d0cbb343988.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.33:44352 10.134.0.33:443 10.133.0.11:46042 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T22:10:27.489478Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:10:27.489528Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:10:27.489548Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:10:27.489554Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-02T22:10:27.164Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 53 - "44.212.242.249,10.133.0.11" "python-requests/2.34.2" "9b65fdcb-8a6d-4cfb-bd88-0573b37c07e3" "maas.apps.1d4172f35d0cbb343988.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.33:443 10.133.0.11:46050 - v1-completions-path [2026-11-02T22:10:27.428Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 61 - "44.212.242.249,10.133.0.11" "python-requests/2.34.2" "8dcbf56a-c209-4b7c-bbe7-dd4eb4feedd2" "maas.apps.1d4172f35d0cbb343988.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.33:33180 10.134.0.33:443 10.133.0.11:46066 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T22:10:28.361592Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:10:28.361625Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:10:28.361637Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:10:28.361642Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-02T22:10:27.811Z] "POST /maas-api/internal/v1/subscriptions/select HTTP/1.1" 404 NR route_not_found - "-" 56 0 8 - "44.212.242.249,10.133.0.11" "python-requests/2.34.2" "6dbfed3a-ff76-4445-9e0a-56281dd9d4de" "maas.apps.1d4172f35d0cbb343988.prod.konflux-ocp-ci.dev" "-" - - 10.134.0.33:443 10.133.0.11:46078 - - [2026-11-02T22:10:28.352Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 91 8 - "44.212.242.249,10.133.0.11" "python-requests/2.34.2" "15ec6138-d875-4754-8ed9-0ea08f5e3f2d" "maas.apps.1d4172f35d0cbb343988.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.33:33180 10.134.0.33:443 10.133.0.11:46084 - odh-ai-gateway-infra.maas-api-route.4 2026-11-02T22:10:29.382345Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-02T22:10:29.309Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 74 - "44.212.242.249,10.133.0.11" "python-requests/2.34.2" "5e574c89-e2c3-4cb1-a83d-b486ad4c5dfa" "maas.apps.1d4172f35d0cbb343988.prod.konflux-ocp-ci.dev" "10.134.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:34652 10.134.0.33:443 10.133.0.11:46090 - v1-completions-path [2026-11-02T22:10:29.558Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 281 86 - "44.212.242.249,10.133.0.11" "python-requests/2.34.2" "7aac5699-6a95-4c0f-a519-6755dffd96e5" "maas.apps.1d4172f35d0cbb343988.prod.konflux-ocp-ci.dev" "10.134.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:34656 10.134.0.33:443 10.133.0.11:46092 - v1-completions-path 2026-11-02T22:10:37.048137Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:10:51.843175Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:13:08.362700Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-02T22:13:08.385402Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:13:08.385458Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:13:08.385481Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T22:13:08.385488Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:13:08.385797Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 2026-11-02T22:13:08.385872Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-02T22:13:08.385881Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-02T22:13:08.385885Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-02T22:13:08.385890Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-11-02T22:13:08.181Z] "POST /v1/completions HTTP/1.1" 404 - via_upstream - "-" 94 126 204 - "44.212.242.249,10.133.0.11" "python-requests/2.34.2" "d92e7304-629e-48ee-9d42-632292e50b48" "maas.apps.1d4172f35d0cbb343988.prod.konflux-ocp-ci.dev" "10.134.0.32:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:34656 10.134.0.33:443 10.133.0.11:44624 - v1-completions-path 2026-11-02T22:14:16.842861Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:14:27.978324Z warn xdsproxy sendDownstreamDelta took 10.496179149s 2026-11-02T22:15:23.318497Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:16:34.664606Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:17:16.247425Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:17:54.479888Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:18:37.989426Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:19:52.226183Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:20:05.044340Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=4 2026-11-02T22:20:18.820913Z warn xdsproxy upstream terminated with unexpected error rpc error: code = Unavailable desc = error reading from server: read tcp 10.134.0.33:38646->172.31.106.197:15012: read: connection reset by peer id=4 2026-11-02T22:20:19.251266Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=5 2026-11-02T22:20:22.873701Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:27:20.231400Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:27:45.943053Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:28:32.023845Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:28:47.443738Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:28:49.038783Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:29:06.332447Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:29:08.012601Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:29:10.653989Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:29:19.103007Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T22:29:22.539Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 72 - "44.212.242.249,10.133.0.11" "python-requests/2.34.2" "41ec07bd-92b7-4537-989b-f6c8de38002d" "maas.apps.1d4172f35d0cbb343988.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:40866 10.134.0.33:443 10.133.0.11:57390 - v1-completions-path [2026-11-02T22:29:22.768Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 62 - "44.212.242.249,10.133.0.11" "python-requests/2.34.2" "c0faa8dc-51df-4a8a-9eaa-1b3f2d428120" "maas.apps.1d4172f35d0cbb343988.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:40866 10.134.0.33:443 10.133.0.11:57400 - v1-completions-path 2026-11-02T22:29:40.323585Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:29:50.405919Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:29:52.011953Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T22:29:58.935Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 61 - "44.212.242.249,10.133.0.11" "python-requests/2.34.2" "6eb6d435-d51b-459e-9edc-336836900065" "maas.apps.1d4172f35d0cbb343988.prod.konflux-ocp-ci.dev" "10.133.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:59482 10.134.0.33:443 10.133.0.11:57086 - v1-completions-path 2026-11-02T22:30:11.037416Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:30:12.631710Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:30:29.178948Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:30:31.299392Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:30:47.881112Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:30:49.616437Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:31:05.575144Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:31:16.310645Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:31:18.963336Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=6 2026-11-02T22:31:19.039386Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:31:41.315435Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:31:56.514806Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:32:06.720267Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:32:10.691701Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:32:10.691736Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:32:10.691750Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T22:32:10.691756Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-02T22:32:10.626Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 65 - "44.212.242.249,10.133.0.11" "python-requests/2.34.2" "8a289095-4e9e-4c36-92f3-eeeef6fcc2d0" "maas.apps.1d4172f35d0cbb343988.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.33:56412 10.134.0.33:443 10.133.0.11:41012 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T22:32:22.156935Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:32:24.441789Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:34:01.019473Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:34:03.363991Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:35:27.842771Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:35:42.555563Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:35:44.278073Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:35:49.675669Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:35:49.675731Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T22:35:49.675747Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T22:35:49.675751Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-02T22:35:49.610Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 65 307 65 - "44.212.242.249,10.133.0.11" "python-requests/2.34.2" "cbe08ba9-add2-4aa4-924d-3ce529898631" "maas.apps.1d4172f35d0cbb343988.prod.konflux-ocp-ci.dev" "10.133.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.33:35068 10.134.0.33:443 10.133.0.11:60484 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T22:36:01.535054Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:37:27.356873Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:37:45.089080Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:38:00.142191Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:38:11.576755Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:38:14.940833Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:38:26.116248Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:38:28.428818Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:38:46.321263Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:41:34.234532Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:41:39.315590Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:41:54.318854Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:41:55.919310Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:42:10.362815Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T22:42:21.065107Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8