2026-10-31T01:07:52.261981Z info FLAG: --concurrency="0" 2026-10-31T01:07:52.261999Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-31T01:07:52.262003Z info FLAG: --help="false" 2026-10-31T01:07:52.262006Z info FLAG: --log_as_json="false" 2026-10-31T01:07:52.262008Z info FLAG: --log_caller="" 2026-10-31T01:07:52.262010Z info FLAG: --log_output_level="default:info" 2026-10-31T01:07:52.262012Z info FLAG: --log_stacktrace_level="default:none" 2026-10-31T01:07:52.262016Z info FLAG: --log_target="[stdout]" 2026-10-31T01:07:52.262019Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-31T01:07:52.262021Z info FLAG: --outlierLogPath="" 2026-10-31T01:07:52.262023Z info FLAG: --profiling="true" 2026-10-31T01:07:52.262025Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-31T01:07:52.262026Z info FLAG: --proxyLogLevel="warning" 2026-10-31T01:07:52.262029Z info FLAG: --serviceCluster="istio-proxy" 2026-10-31T01:07:52.262030Z info FLAG: --stsPort="0" 2026-10-31T01:07:52.262032Z info FLAG: --templateFile="" 2026-10-31T01:07:52.262034Z info FLAG: --tokenManagerPlugin="" 2026-10-31T01:07:52.262037Z info FLAG: --vklog="0" 2026-10-31T01:07:52.262039Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-31T01:07:52.262170Z info Proxy role ips=[10.133.0.48] type=router id=maas-default-gateway-openshift-default-855f9574cf-qhwdc.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-31T01:07:52.262227Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-31T01:07:52.263339Z info cpu limit detected as 2, setting concurrency 2026-10-31T01:07:52.263556Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-31T01:07:52.263567Z info JWT policy is third-party-jwt 2026-10-31T01:07:52.263570Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-31T01:07:52.265437Z info Starting default Istio SDS Server 2026-10-31T01:07:52.265463Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-31T01:07:52.265621Z info Opening status port 15020 2026-10-31T01:07:52.266097Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-31T01:07:52.266725Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-31T01:07:52.268067Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-31T01:07:52.268812Z info Starting proxy agent 2026-10-31T01:07:52.268840Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-31T01:07:52.278160Z info sds Starting SDS grpc server 2026-10-31T01:07:52.278176Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-31T01:07:52.330789Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-31T01:07:52.333104Z info cache generated new workload certificate resourceName=default latency=66.190711ms ttl=23h59m59.6668992s 2026-10-31T01:07:52.333135Z info cache Root cert has changed, start rotating root cert 2026-10-31T01:07:52.333164Z info cache returned workload trust anchor from cache ttl=23h59m59.666837172s 2026-10-31T01:07:52.450558Z info ads ADS: new connection for node:1 2026-10-31T01:07:52.450616Z info cache returned workload certificate from cache ttl=23h59m59.54938504s 2026-10-31T01:07:52.451040Z info ads ADS: new connection for node:2 2026-10-31T01:07:52.451106Z info cache returned workload trust anchor from cache ttl=23h59m59.548894787s 2026-10-31T01:07:52.532770Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:08:00.358346Z info Readiness succeeded in 8.098594698s 2026-10-31T01:08:00.358690Z info Envoy proxy is ready 2026-10-31T01:08:04.528321Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:08:12.493739Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:08:13.078065Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-31T01:08:48.485Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 359 84 - "52.71.36.72,10.132.0.19" "curl/7.76.1" "086b9a50-86ac-43a6-ad53-1475ba3ce812" "maas.apps.c2ed91a9a00e3dda98de.prod.konflux-ocp-ci.dev" "10.134.0.36:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:42302 10.133.0.48:443 10.132.0.19:43960 - v1-model-routing [2026-10-31T01:08:48.634Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 375 63 - "52.71.36.72,10.132.0.19" "curl/7.76.1" "667ebe79-a482-43b9-93b4-3916bebeafe0" "maas.apps.c2ed91a9a00e3dda98de.prod.konflux-ocp-ci.dev" "10.134.0.36:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:42310 10.133.0.48:443 10.132.0.19:43970 - v1-model-routing [2026-10-31T01:08:48.922Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 52 - "52.71.36.72,10.132.0.19" "curl/7.76.1" "72237711-92f1-4280-85d4-6148e028f8a6" "maas.apps.c2ed91a9a00e3dda98de.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.48:443 10.132.0.19:43990 - v1-model-routing 2026-10-31T01:11:08.315932Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T01:11:08.315972Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T01:11:08.315986Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T01:11:08.315991Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T01:11:08.546532Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T01:11:08.546566Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T01:11:08.546580Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T01:11:08.546585Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T01:11:09.194860Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T01:11:09.194896Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T01:11:09.194910Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T01:11:09.194915Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-31T01:11:08.251Z] "DELETE /v1/api-keys/1ff31dee-88cf-4372-a4ad-4dd336e98e2c HTTP/1.1" 200 - via_upstream - "-" 0 321 64 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "7d4e8738-5232-4fbf-b971-0fd88f51cd2c" "maas.apps.c2ed91a9a00e3dda98de.prod.konflux-ocp-ci.dev" "10.132.0.38:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.48:35532 10.133.0.48:443 10.132.0.19:60950 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T01:11:08.374Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 44 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "ed38cf1b-cf06-4ea7-bf5d-91e82d47e9bb" "maas.apps.c2ed91a9a00e3dda98de.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.133.0.48:443 10.132.0.19:60952 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T01:11:08.482Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 63 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "3aefaaf6-75b8-4472-a9fe-3c59e759afcc" "maas.apps.c2ed91a9a00e3dda98de.prod.konflux-ocp-ci.dev" "10.132.0.38:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.48:35538 10.133.0.48:443 10.132.0.19:60962 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T01:11:09.051Z] "DELETE /v1/api-keys/9a86b7f1-d7d6-4a5f-80e0-cfc234fcebe0 HTTP/1.1" 200 - via_upstream - "-" 0 321 143 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "7be1c8db-b7bb-4d65-9510-d2ee4e471215" "maas.apps.c2ed91a9a00e3dda98de.prod.konflux-ocp-ci.dev" "10.132.0.38:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.48:35538 10.133.0.48:443 10.132.0.19:60980 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T01:11:10.177Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 37 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "205d1481-28fb-49a9-9b4c-efd998195955" "maas.apps.c2ed91a9a00e3dda98de.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.133.0.48:443 10.132.0.19:59418 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T01:11:10.243Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 38 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "48d0b43a-5e1e-4c5e-bec8-45082b15a446" "maas.apps.c2ed91a9a00e3dda98de.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.133.0.48:443 10.132.0.19:59432 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T01:11:10.438047Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T01:11:10.438101Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T01:11:10.438120Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T01:11:10.438127Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T01:11:10.983888Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T01:11:10.983920Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T01:11:10.983934Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T01:11:10.983938Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-31T01:11:10.380Z] "DELETE /v1/api-keys/9b428971-6e7b-4a76-99b4-773cca6aea94 HTTP/1.1" 200 - via_upstream - "-" 0 321 57 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "3c95ad53-8193-4724-af05-ed7ca524d4b2" "maas.apps.c2ed91a9a00e3dda98de.prod.konflux-ocp-ci.dev" "10.132.0.38:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.48:35532 10.133.0.48:443 10.132.0.19:59438 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T01:11:10.925Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 317 58 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "c3634fbe-886d-4572-9557-adc3c396c8a3" "maas.apps.c2ed91a9a00e3dda98de.prod.konflux-ocp-ci.dev" "10.132.0.38:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.48:35538 10.133.0.48:443 10.132.0.19:59442 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T01:11:17.416Z] "POST /maas-api/internal/v1/api-keys/cleanup HTTP/1.1" 404 NR route_not_found - "-" 2 0 18 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "45432e66-d0d8-4690-bbb3-5e66bf2463ac" "maas.apps.c2ed91a9a00e3dda98de.prod.konflux-ocp-ci.dev" "-" - - 10.133.0.48:443 10.132.0.19:59472 - - 2026-10-31T01:11:30.963723Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:11:47.451618Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:15:35.990950Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:15:58.250977Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:16:45.166356Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:17:00.959205Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:17:02.683648Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:17:19.851547Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:17:21.876895Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:17:25.261263Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:17:32.498754Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:17:40.335283Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-10-31T01:17:40.282Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 69 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "8a927d58-11e8-419b-ac78-0ce47d55ccc6" "maas.apps.c2ed91a9a00e3dda98de.prod.konflux-ocp-ci.dev" "10.134.0.35:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:57370 10.133.0.48:443 10.132.0.19:49876 - v1-completions-path 2026-10-31T01:17:57.047972Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:18:07.283880Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:18:09.026298Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-31T01:18:14.741Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 60 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "5ef8455d-ad45-4db1-9802-2a634abd9673" "maas.apps.c2ed91a9a00e3dda98de.prod.konflux-ocp-ci.dev" "10.134.0.35:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:57370 10.133.0.48:443 10.132.0.19:53868 - v1-completions-path [2026-10-31T01:18:15.574Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 57 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "3d2b9a26-fdca-4c8d-be8e-129f266f15c9" "maas.apps.c2ed91a9a00e3dda98de.prod.konflux-ocp-ci.dev" "10.134.0.35:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:57370 10.133.0.48:443 10.132.0.19:53880 - v1-completions-path 2026-10-31T01:18:28.023776Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:18:29.676769Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:18:46.101259Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:18:48.293378Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:19:04.853644Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:19:06.554336Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:19:22.692131Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:19:33.216997Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:19:36.411801Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:19:52.000909Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:19:54.178518Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:19:57.873728Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-10-31T01:19:57.873836Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T01:19:57.873858Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T01:19:57.873874Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T01:19:57.873879Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-31T01:19:57.485Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 59 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "7d3e9b73-2c00-47fa-9bb2-208b3dd0021f" "maas.apps.c2ed91a9a00e3dda98de.prod.konflux-ocp-ci.dev" "10.134.0.36:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:42302 10.133.0.48:443 10.132.0.19:47820 - v1-completions-path [2026-10-31T01:19:57.602Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 32 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "ad6fc17f-5f65-4c9a-b7fa-16b1b1da4c93" "maas.apps.c2ed91a9a00e3dda98de.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.48:443 10.132.0.19:47824 - v1-completions-path [2026-10-31T01:19:57.813Z] "DELETE /v1/api-keys/3086d962-84e8-4371-8462-cf8d769a4534 HTTP/1.1" 200 - via_upstream - "-" 0 375 60 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "bfeb5294-d3a9-48aa-9d1f-86d10cd56ba5" "maas.apps.c2ed91a9a00e3dda98de.prod.konflux-ocp-ci.dev" "10.132.0.38:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.48:39648 10.133.0.48:443 10.132.0.19:47836 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T01:20:01.754786Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:20:20.022618Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:20:30.239012Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-31T01:20:45.141Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 36 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "b4c925ac-68f2-40c3-9032-608d22891a84" "maas.apps.c2ed91a9a00e3dda98de.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.48:443 10.132.0.19:53120 - v1-completions-path 2026-10-31T01:20:46.132358Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:20:48.383719Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:22:25.528894Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:22:27.999257Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:23:53.906298Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:24:08.907991Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:24:10.574979Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:24:27.697733Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:25:51.505040Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T01:25:51.505078Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T01:25:51.505090Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T01:25:51.505095Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-31T01:25:51.423Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 81 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "69b24c95-0c34-40ad-8f7c-dcc2dc399815" "maas.apps.c2ed91a9a00e3dda98de.prod.konflux-ocp-ci.dev" "10.132.0.38:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.48:47948 10.133.0.48:443 10.132.0.19:53904 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T01:25:56.049127Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:26:11.392008Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:26:27.819317Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:26:39.878084Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:26:43.583859Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:26:52.355738Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:26:54.943281Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:27:12.860356Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:30:00.348598Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T01:30:00.348624Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T01:30:00.348627Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T01:30:00.348632Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-10-31T01:30:00.288Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 403 - - "-" 61 103 60 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "5e29b390-e337-4dfb-b7aa-f88750552430" "maas.apps.c2ed91a9a00e3dda98de.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.48:443 10.132.0.19:58094 - v1-catch-all-path 2026-10-31T01:30:02.271006Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:30:05.468190Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 2026-10-31T01:30:05.587007Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-10-31T01:30:05.381Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 86 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "18d8171e-0ee3-4529-8bab-ef6c83789ab5" "maas.apps.c2ed91a9a00e3dda98de.prod.konflux-ocp-ci.dev" "10.133.0.46:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:40176 10.133.0.48:443 10.132.0.19:58120 - v1-catch-all-path [2026-10-31T01:30:05.525Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 91 3616 61 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "ef69123b-2305-4d99-9f1a-e0c1dd52052e" "maas.apps.c2ed91a9a00e3dda98de.prod.konflux-ocp-ci.dev" "10.133.0.46:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:40192 10.133.0.48:443 10.132.0.19:58122 - v1-catch-all-model-routing 2026-10-31T01:30:07.469953Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:30:22.807572Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:30:24.340197Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:30:38.699664Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-31T01:30:49.450753Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9