2026-11-03T22:21:28.998122Z info FLAG: --concurrency="0" 2026-11-03T22:21:28.998140Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T22:21:28.998144Z info FLAG: --help="false" 2026-11-03T22:21:28.998147Z info FLAG: --log_as_json="false" 2026-11-03T22:21:28.998149Z info FLAG: --log_caller="" 2026-11-03T22:21:28.998151Z info FLAG: --log_output_level="default:info" 2026-11-03T22:21:28.998153Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T22:21:28.998158Z info FLAG: --log_target="[stdout]" 2026-11-03T22:21:28.998160Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T22:21:28.998162Z info FLAG: --outlierLogPath="" 2026-11-03T22:21:28.998164Z info FLAG: --profiling="true" 2026-11-03T22:21:28.998166Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T22:21:28.998169Z info FLAG: --proxyLogLevel="warning" 2026-11-03T22:21:28.998171Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T22:21:28.998173Z info FLAG: --stsPort="0" 2026-11-03T22:21:28.998175Z info FLAG: --templateFile="" 2026-11-03T22:21:28.998177Z info FLAG: --tokenManagerPlugin="" 2026-11-03T22:21:28.998181Z info FLAG: --vklog="0" 2026-11-03T22:21:28.998183Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T22:21:28.998369Z info Proxy role ips=[10.132.0.45] type=router id=maas-default-gateway-openshift-default-7df854bf67-xt8wx.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T22:21:28.998413Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T22:21:28.999436Z info cpu limit detected as 2, setting concurrency 2026-11-03T22:21:28.999637Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T22:21:28.999647Z info JWT policy is third-party-jwt 2026-11-03T22:21:28.999650Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T22:21:29.001498Z info Starting default Istio SDS Server 2026-11-03T22:21:29.001527Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T22:21:29.001560Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T22:21:29.002067Z info Opening status port 15020 2026-11-03T22:21:29.002946Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T22:21:29.003568Z info sds Starting SDS grpc server 2026-11-03T22:21:29.003601Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T22:21:29.004337Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T22:21:29.005364Z info Starting proxy agent 2026-11-03T22:21:29.005393Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T22:21:29.069730Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T22:21:29.089966Z info cache generated new workload certificate resourceName=default latency=86.283053ms ttl=23h59m59.910036547s 2026-11-03T22:21:29.089994Z info cache Root cert has changed, start rotating root cert 2026-11-03T22:21:29.090023Z info cache returned workload trust anchor from cache ttl=23h59m59.909978116s 2026-11-03T22:21:29.189107Z info ads ADS: new connection for node:1 2026-11-03T22:21:29.189163Z info cache returned workload certificate from cache ttl=23h59m59.810838897s 2026-11-03T22:21:29.189821Z info ads ADS: new connection for node:2 2026-11-03T22:21:29.189912Z info cache returned workload trust anchor from cache ttl=23h59m59.810089183s 2026-11-03T22:21:29.272087Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:21:37.185764Z info Readiness succeeded in 8.190620828s 2026-11-03T22:21:37.186201Z info Envoy proxy is ready 2026-11-03T22:25:10.030910Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:25:10.030951Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:25:10.030968Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:25:10.030975Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T22:25:09.894Z] "DELETE /v1/api-keys/eee65512-9f3d-4091-ad5d-be3a13f16ba5 HTTP/1.1" 200 - via_upstream - "-" 0 370 136 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "efb7433f-c5ae-4104-8dac-f7100733c828" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.133.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.45:52142 10.132.0.45:443 10.132.0.18:49224 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T22:25:10.947588Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:25:10.947621Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:25:10.947632Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:25:10.947637Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T22:25:10.849Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 98 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "7b4c428f-21a6-46ba-bead-baa442668636" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.133.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.45:52152 10.132.0.45:443 10.132.0.18:49232 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T22:25:10.974Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 74 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "35a0562b-9b9b-4fa3-af2f-125362b7fd6a" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.132.0.45:443 10.132.0.18:49234 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T22:25:16.560Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 93 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "eb86f111-93ec-4394-8c73-afe072342377" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.132.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:42566 10.132.0.45:443 10.132.0.18:55030 - v1-completions-path 2026-11-03T22:25:21.254160Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:25:31.702212Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:29:54.185840Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:30:40.312667Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:31:20.446519Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:32:37.755259Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:33:21.772402Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:35:09.636604Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:40:09.517275Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:41:53.983594Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:54:02.651022Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-03T23:01:26.705825Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:01:26.705867Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:01:26.705884Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T23:01:26.705890Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:01:26.706105Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 2026-11-03T23:01:26.706173Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T23:01:26.706184Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T23:01:26.706188Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T23:01:26.706194Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-11-03T23:01:26.637Z] "POST /v1/chat/completions HTTP/1.1" 404 - via_upstream - "-" 120 126 68 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "e658484f-ad96-475e-b142-37d40813cdb3" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.132.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:42566 10.132.0.45:443 10.132.0.18:58838 - v1-chat-completions-path 2026-11-03T23:07:50.874381Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:08:14.628153Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:09:05.875389Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:09:09.783103Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:09:09.783141Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:09:09.783157Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:09:09.783165Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T23:09:09.710Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 87 329 72 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "f73c9d7f-a849-4af5-a3dd-6e7862924afb" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.133.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.45:60504 10.132.0.45:443 10.132.0.18:56348 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T23:09:10.004Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 73 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "874eca5f-5732-4975-8ef3-79fc304d3771" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.132.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:44200 10.132.0.45:443 10.132.0.18:56356 - v1-completions-path [2026-11-03T23:09:10.587Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 59 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "fd0a27e7-0041-4894-9908-b1ded21a9ecd" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.132.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:44216 10.132.0.45:443 10.132.0.18:56372 - v1-completions-path 2026-11-03T23:09:22.849202Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:09:27.453761Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T23:09:30.371Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 57 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "195e7b61-4dfd-46ae-98f8-10792bd018fe" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.132.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:44216 10.132.0.45:443 10.132.0.18:53334 - v1-completions-path 2026-11-03T23:09:42.021004Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-11-03T23:09:41.906Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 287 115 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "bf8f2c4b-5a10-43ca-8e3a-62b1057202eb" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.132.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:42566 10.132.0.45:443 10.132.0.18:60768 - v1-completions-path 2026-11-03T23:09:45.054831Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:09:58.867120Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:10:03.620427Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:10:03.620463Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:10:03.620479Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:10:03.620485Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T23:10:03.551Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 324 69 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "e77fcc96-a342-4adc-af79-8a9559c05556" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.133.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.45:38544 10.132.0.45:443 10.132.0.18:37632 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T23:10:05.039Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 63 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "d0c2928a-51b1-4823-98fc-ebb13335f03e" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.132.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:44200 10.132.0.45:443 10.132.0.18:37640 - v1-completions-path 2026-11-03T23:10:05.621460Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-03T23:10:05.569Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 55 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "a37a4fe7-a462-494e-9120-2b4cb1da8eab" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.132.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:44216 10.132.0.45:443 10.132.0.18:37656 - v1-completions-path [2026-11-03T23:10:06.135Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 55 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "0b3b1854-f518-46ee-9fa5-1b64f8db0fb1" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.132.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:44216 10.132.0.45:443 10.132.0.18:37666 - v1-completions-path [2026-11-03T23:10:06.793Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 58 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "ff496245-ff19-4f8d-9f31-5633e0f27cad" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.132.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:44216 10.132.0.45:443 10.132.0.18:37674 - v1-completions-path 2026-11-03T23:10:22.181317Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:10:32.584539Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:10:37.610126Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:10:40.912458Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:10:40.912491Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:10:40.912504Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T23:10:40.912509Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T23:10:40.845Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 317 67 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "de79b84c-3029-4355-857a-011f69bff7f3" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.133.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.45:38544 10.132.0.45:443 10.132.0.18:37538 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T23:10:41.157Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 58 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "d075c257-1d34-4fba-9253-c908521c06d4" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.132.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:44200 10.132.0.45:443 10.132.0.18:37564 - v1-completions-path [2026-11-03T23:10:41.246Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 57 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "70d304ed-b30d-4ad8-8db3-6990596bbabf" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.132.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:44200 10.132.0.45:443 10.132.0.18:37576 - v1-completions-path 2026-11-03T23:10:54.249780Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:10:56.603870Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:11:16.656474Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T23:11:19.418Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 63 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "c1796f9f-6ab9-49f9-bccc-446c8601e914" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.132.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:45416 10.132.0.45:443 10.132.0.18:59520 - v1-completions-path 2026-11-03T23:11:21.952761Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:11:38.809079Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:11:54.829607Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:12:05.659015Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:12:08.209462Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:12:26.729853Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:12:29.697844Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T23:12:34.737Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 62 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "812b26ef-15e2-4c67-9fe8-8fbb3cf39f0c" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.132.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:45416 10.132.0.45:443 10.132.0.18:58766 - v1-completions-path [2026-11-03T23:12:36.326Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 76 0 33 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "7b0744b4-2a47-43a5-8fa1-fe8bababae9b" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.45:443 10.132.0.18:58774 - v1-completions-path 2026-11-03T23:12:41.912078Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:12:47.358046Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:12:47.358087Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:12:47.358105Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T23:12:47.358111Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T23:12:47.286Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 65 307 71 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "b79ac228-0810-47a9-abab-5c0ecadebbcc" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.133.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.45:43882 10.132.0.45:443 10.132.0.18:56758 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T23:12:59.702921Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:13:09.985540Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T23:13:27.795Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 35 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "5f6ee5da-cd43-4f12-91a7-99ea2f4515ef" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.45:443 10.132.0.18:51260 - v1-completions-path 2026-11-03T23:13:30.023562Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:13:33.510094Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:15:16.976877Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:15:19.795966Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:16:49.627849Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:17:05.178218Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:17:07.500001Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:17:26.211246Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:18:55.708222Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:19:13.489305Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:19:32.553778Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:19:44.695039Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:19:48.514560Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:19:58.654620Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:20:03.213605Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T23:20:08.654Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 62 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "7cc9356e-7ecb-4d4b-a844-093b8778a831" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.132.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:44200 10.132.0.45:443 10.132.0.18:52480 - v1-completions-path 2026-11-03T23:20:21.078358Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:23:09.816793Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T23:23:09.816816Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T23:23:09.816819Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-03T23:23:09.816823Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-11-03T23:23:09.740Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 403 - - "-" 61 103 76 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "1c398e0b-c427-43b1-89fd-d9ba0c266839" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.45:443 10.132.0.18:45096 - v1-catch-all-path 2026-11-03T23:23:12.330565Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:23:20.462408Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:23:25.901749Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-11-03T23:23:25.826Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 75 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "c1a68036-b20e-4e2f-82a6-916c5d2bd945" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.132.0.38:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:49494 10.132.0.45:443 10.132.0.18:41466 - v1-catch-all-path 2026-11-03T23:23:38.450719Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:23:40.887601Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:23:43.375763Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:23:43.375812Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T23:23:43.375833Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T23:23:43.375841Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T23:23:43.264Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 73 316 111 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "b7cf462c-05fb-4342-bf61-4ff5a18d240a" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.133.0.196:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.45:57640 10.132.0.45:443 10.132.0.18:50248 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T23:23:43.457529Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-11-03T23:23:43.395Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 61 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "fa82d54f-dd34-4940-9221-814a1acd1d96" "maas.apps.8d27f8ca2de81131e1aa.prod.konflux-ocp-ci.dev" "10.132.0.38:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:49494 10.132.0.45:443 10.132.0.18:36646 - v1-catch-all-path 2026-11-03T23:23:55.502090Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:24:06.411408Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T23:24:32.500473Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=3