2026-10-31T16:09:16.015755Z info FLAG: --concurrency="0" 2026-10-31T16:09:16.015773Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-31T16:09:16.015777Z info FLAG: --help="false" 2026-10-31T16:09:16.015779Z info FLAG: --log_as_json="false" 2026-10-31T16:09:16.015782Z info FLAG: --log_caller="" 2026-10-31T16:09:16.015783Z info FLAG: --log_output_level="default:info" 2026-10-31T16:09:16.015786Z info FLAG: --log_stacktrace_level="default:none" 2026-10-31T16:09:16.015790Z info FLAG: --log_target="[stdout]" 2026-10-31T16:09:16.015792Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-31T16:09:16.015794Z info FLAG: --outlierLogPath="" 2026-10-31T16:09:16.015796Z info FLAG: --profiling="true" 2026-10-31T16:09:16.015798Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-31T16:09:16.015800Z info FLAG: --proxyLogLevel="warning" 2026-10-31T16:09:16.015802Z info FLAG: --serviceCluster="istio-proxy" 2026-10-31T16:09:16.015804Z info FLAG: --stsPort="0" 2026-10-31T16:09:16.015807Z info FLAG: --templateFile="" 2026-10-31T16:09:16.015810Z info FLAG: --tokenManagerPlugin="" 2026-10-31T16:09:16.015814Z info FLAG: --vklog="0" 2026-10-31T16:09:16.015817Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-31T16:09:16.015963Z info Proxy role ips=[10.132.0.34] type=router id=maas-default-gateway-openshift-default-fcb75c45f-jdhp6.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-31T16:09:16.016012Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-31T16:09:16.017116Z info cpu limit detected as 2, setting concurrency 2026-10-31T16:09:16.017404Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-31T16:09:16.017417Z info JWT policy is third-party-jwt 2026-10-31T16:09:16.017420Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-31T16:09:16.019592Z info Starting default Istio SDS Server 2026-10-31T16:09:16.019619Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-31T16:09:16.019649Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-31T16:09:16.020079Z info Opening status port 15020 2026-10-31T16:09:16.021597Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-31T16:09:16.022163Z info sds Starting SDS grpc server 2026-10-31T16:09:16.022431Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-31T16:09:16.022973Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-31T16:09:16.023777Z info Starting proxy agent 2026-10-31T16:09:16.023806Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-31T16:09:16.093748Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-31T16:09:16.244092Z info ads ADS: new connection for node:1 2026-10-31T16:09:16.244436Z info ads ADS: new connection for node:2 2026-10-31T16:09:16.270870Z info cache generated new workload certificate resourceName=default latency=248.336289ms ttl=23h59m59.72913337s 2026-10-31T16:09:16.270916Z info cache Root cert has changed, start rotating root cert 2026-10-31T16:09:16.270944Z info cache returned workload certificate from cache ttl=23h59m59.729059027s 2026-10-31T16:09:16.270956Z info cache returned workload trust anchor from cache ttl=23h59m59.72904458s 2026-10-31T16:09:16.271047Z info cache returned workload trust anchor from cache ttl=23h59m59.728954212s 2026-10-31T16:09:16.271431Z info cache returned workload trust anchor from cache ttl=23h59m59.728570171s 2026-10-31T16:09:16.317091Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:09:21.914972Z info Readiness succeeded in 5.901945223s 2026-10-31T16:09:21.915312Z info Envoy proxy is ready 2026-10-31T16:10:54.253797Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:10:58.338296Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:11:04.117647Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:11:13.080149Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:11:22.334181Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:11:50.781516Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:11:53.287293Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:12:26.853868Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T16:12:26.853914Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T16:12:26.853934Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T16:12:26.853941Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-31T16:12:26.677Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 320 176 - "34.228.250.194,10.132.0.9" "curl/7.76.1" "75679c52-8da9-4bbd-904f-544bfc493e2b" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "10.132.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:44672 10.132.0.34:443 10.132.0.9:36902 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T16:12:29.156Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 56 - "34.228.250.194,10.132.0.9" "curl/7.76.1" "6cb89ffe-2909-48d6-91fd-cdabc4e22321" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.34:443 10.132.0.9:36916 - v1-model-routing 2026-10-31T16:17:17.582204Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T16:17:17.582239Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T16:17:17.582253Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T16:17:17.582258Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-31T16:17:17.458Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 84 326 123 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "3e926da5-1619-4bb8-817a-0e0892d82060" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "10.132.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:48260 10.132.0.34:443 10.132.0.9:44852 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T16:17:21.139Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 91 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "681a4194-6ff0-4fa7-a95a-8c0f3c0c43e5" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.132.0.34:443 10.132.0.9:51236 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T16:17:21.479591Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T16:17:21.479629Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T16:17:21.479643Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T16:17:21.479648Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T16:17:22.257772Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T16:17:22.257826Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T16:17:22.257848Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T16:17:22.257855Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-31T16:17:21.371Z] "DELETE /v1/api-keys/ee5a99c2-fe5f-4d47-a110-106481193a18 HTTP/1.1" 200 - via_upstream - "-" 0 321 107 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "5eda7eea-fcf7-4ca4-b17e-a11e4045e6be" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "10.132.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:48748 10.132.0.34:443 10.132.0.9:51248 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T16:17:22.154Z] "DELETE /v1/api-keys/98c2bfaf-c616-4b12-ba63-35dbf9b235e0 HTTP/1.1" 200 - via_upstream - "-" 0 321 103 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "402041df-98a6-4012-9aa7-7d4df2ea25f0" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "10.132.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:48748 10.132.0.34:443 10.132.0.9:51264 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T16:17:22.790Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 89 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "d06b4ed3-39b9-41b2-9c4b-16cd364231c7" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.132.0.34:443 10.132.0.9:51266 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T16:17:23.601266Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T16:17:23.601299Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T16:17:23.601313Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T16:17:23.601318Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T16:17:23.823688Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T16:17:23.823712Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T16:17:23.823715Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T16:17:23.823719Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-10-31T16:17:23.497Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 103 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "9ad3d81e-3c6e-41e0-9bbe-b45df910955b" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "10.132.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:48748 10.132.0.34:443 10.132.0.9:51278 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T16:17:23.789Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 77 34 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "623d7d21-3bfe-4028-937e-c3399c9ee4f8" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.34:443 10.132.0.9:51298 - v1-completions-path [2026-10-31T16:17:30.656Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 290 79 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "10e4ed1f-4168-4db9-a861-56c6695b6207" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:41094 10.132.0.34:443 10.132.0.9:39770 - v1-completions-path [2026-10-31T16:17:32.148Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 279 71 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "7bfcc9f1-daea-437b-a97e-3bc155ad9be0" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:41110 10.132.0.34:443 10.132.0.9:39786 - v1-completions-path 2026-10-31T16:17:32.900811Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-10-31T16:17:32.905589Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T16:17:32.905647Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T16:17:32.905670Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T16:17:32.905677Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-31T16:17:32.800Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 105 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "f2da5cc8-0a4e-45c7-8060-f64c966c5676" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "10.132.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:48260 10.132.0.34:443 10.132.0.9:39792 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T16:17:42.471844Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:18:00.277753Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T16:23:52.149Z] "GET /v1/tenants HTTP/1.1" 404 NR route_not_found - "-" 0 0 17 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "3c17dc89-6222-447e-9290-515da33c125b" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "-" - - 10.132.0.34:443 10.132.0.9:53812 - - 2026-10-31T16:24:38.878957Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:26:11.918032Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:26:45.072372Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:28:27.275265Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:39:19.269507Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:39:54.622813Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-10-31T16:40:41.958222Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:41:13.412386Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:42:50.156865Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:48:46.861699Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T16:48:46.861733Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T16:48:46.861747Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T16:48:46.861752Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T16:48:47.003782Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T16:48:46.754Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 107 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "143b1535-7854-4501-83e7-8ae2b926a8a4" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "10.132.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:52350 10.132.0.34:443 10.132.0.9:54130 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T16:49:14.976755Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:49:24.930166Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T16:49:24.930201Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T16:49:24.930213Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T16:49:24.930217Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-31T16:49:24.819Z] "POST /v1/api-keys HTTP/1.1" 400 - via_upstream - "-" 80 94 110 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "106d5f17-ce18-43f2-a719-9001c0a836ff" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "10.132.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:53306 10.132.0.34:443 10.132.0.9:43204 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T16:50:01.168217Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-10-31T16:50:01.176182Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T16:50:01.176221Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T16:50:01.176241Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T16:50:01.176248Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-31T16:50:01.062Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 74 316 114 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "92c9cf5a-45af-4f30-a122-3d50bdb4d7c1" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "10.132.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:53306 10.132.0.34:443 10.132.0.9:38246 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T16:50:11.382590Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T16:50:17.605Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 73 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "45b9f3b7-2ddc-4c9e-a9d0-cff930db3d62" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:46022 10.132.0.34:443 10.132.0.9:54700 - v1-completions-path [2026-10-31T16:50:18.359Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 58 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "362663dc-5c61-4825-8969-1083785fa2af" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:46022 10.132.0.34:443 10.132.0.9:54708 - v1-completions-path 2026-10-31T16:50:32.848399Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:50:41.201175Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:51:04.170406Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:51:22.994238Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T16:51:31.395Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 297 68 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "031daea1-a069-4ad3-9b99-4d463f97ecdf" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:56756 10.132.0.34:443 10.132.0.9:58970 - v1-completions-path [2026-10-31T16:51:36.418Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 59 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "151364fb-31b0-4122-858f-0422665bbb0c" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:56756 10.132.0.34:443 10.132.0.9:58994 - v1-completions-path 2026-10-31T16:51:54.754731Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:52:05.575257Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:52:12.035906Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T16:52:24.744Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 63 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "1e121ab3-50ad-48e7-92ca-c5f84481aa92" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:46022 10.132.0.34:443 10.132.0.9:51966 - v1-completions-path 2026-10-31T16:52:40.615640Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:52:45.275061Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:53:08.658104Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:53:15.030030Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:53:41.404405Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:54:03.506798Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:54:16.654926Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:54:22.774942Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T16:54:34.993Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 65 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "acabd04f-9472-48e8-9b7e-f3c3080e3a28" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:41110 10.132.0.34:443 10.132.0.9:46326 - v1-completions-path 2026-10-31T16:54:50.509819Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:54:56.016998Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:55:14.578371Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T16:55:22.443Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 59 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "67d82e54-149c-485e-acd1-a98f786f1ad8" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "10.133.0.29:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:46022 10.132.0.34:443 10.132.0.9:48016 - v1-completions-path 2026-10-31T16:55:36.356096Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:55:46.890010Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T16:56:07.120Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 39 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "4321e4f3-c3ff-4a08-9879-7f0d03df0035" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.34:443 10.132.0.9:38030 - v1-completions-path 2026-10-31T16:56:11.382553Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:56:17.207163Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:58:40.325197Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T16:58:46.210419Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:00:25.375859Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:00:42.327386Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:00:46.602479Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:00:57.334462Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T17:00:57.334496Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T17:00:57.334509Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T17:00:57.334513Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-31T17:00:57.221Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 65 307 112 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "849c19df-8cd4-4051-bac2-67447a69a293" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "10.132.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:34356 10.132.0.34:443 10.132.0.9:45666 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T17:00:57.535Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 61 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "4c34bc57-2bf5-4be3-b7a1-bdcd7007c9d4" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:41110 10.132.0.34:443 10.132.0.9:45670 - v1-completions-path 2026-10-31T17:01:11.510627Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:03:18.345750Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:03:58.813390Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:04:26.206223Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:04:41.548323Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:04:47.980762Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:05:01.006649Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:05:05.698169Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:05:17.157858Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T17:05:17.157894Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T17:05:17.157909Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T17:05:17.157914Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-31T17:05:17.032Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 336 125 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "8cad101c-3f2d-45f3-9882-e976f4dc00e0" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "10.132.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:35444 10.132.0.34:443 10.132.0.9:42302 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T17:05:17.410598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T17:05:17.410642Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T17:05:17.410665Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T17:05:17.410671Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-31T17:05:17.362Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 524 48 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "27a3476b-1e78-4f45-8395-b27c6fef5851" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "10.132.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.34:35444 10.132.0.34:443 10.132.0.9:42314 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T17:05:32.092121Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:08:39.302563Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=3 2026-10-31T17:09:09.341215Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:09:20.508256Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:09:27.019528Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-10-31T17:09:26.942Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 77 - "34.228.250.194,10.132.0.9" "python-requests/2.34.2" "fdf8116a-adfe-4813-a2a3-25546b4de049" "maas.apps.57d06e07271cfd892499.prod.konflux-ocp-ci.dev" "10.134.0.32:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.34:49968 10.132.0.34:443 10.132.0.9:55960 - v1-catch-all-path 2026-10-31T17:09:41.461219Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:09:45.907975Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:10:02.317082Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T17:10:14.125685Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8