2026-11-03T21:56:49.630572Z info FLAG: --concurrency="0" 2026-11-03T21:56:49.630588Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T21:56:49.630592Z info FLAG: --help="false" 2026-11-03T21:56:49.630594Z info FLAG: --log_as_json="false" 2026-11-03T21:56:49.630596Z info FLAG: --log_caller="" 2026-11-03T21:56:49.630598Z info FLAG: --log_output_level="default:info" 2026-11-03T21:56:49.630600Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T21:56:49.630605Z info FLAG: --log_target="[stdout]" 2026-11-03T21:56:49.630607Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T21:56:49.630608Z info FLAG: --outlierLogPath="" 2026-11-03T21:56:49.630610Z info FLAG: --profiling="true" 2026-11-03T21:56:49.630612Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T21:56:49.630614Z info FLAG: --proxyLogLevel="warning" 2026-11-03T21:56:49.630616Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T21:56:49.630618Z info FLAG: --stsPort="0" 2026-11-03T21:56:49.630619Z info FLAG: --templateFile="" 2026-11-03T21:56:49.630621Z info FLAG: --tokenManagerPlugin="" 2026-11-03T21:56:49.630628Z info FLAG: --vklog="0" 2026-11-03T21:56:49.630630Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T21:56:49.630758Z info Proxy role ips=[10.134.0.43] type=router id=maas-default-gateway-openshift-default-66db998887-7z9gp.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T21:56:49.630801Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T21:56:49.631821Z info cpu limit detected as 2, setting concurrency 2026-11-03T21:56:49.632021Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T21:56:49.632033Z info JWT policy is third-party-jwt 2026-11-03T21:56:49.632036Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T21:56:49.634206Z info Starting default Istio SDS Server 2026-11-03T21:56:49.634240Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T21:56:49.634255Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T21:56:49.635116Z info Opening status port 15020 2026-11-03T21:56:49.635579Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T21:56:49.636146Z info sds Starting SDS grpc server 2026-11-03T21:56:49.636161Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T21:56:49.636920Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T21:56:49.637836Z info Starting proxy agent 2026-11-03T21:56:49.637867Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T21:56:49.690196Z info cache generated new workload certificate resourceName=default latency=53.921309ms ttl=23h59m59.309806736s 2026-11-03T21:56:49.690246Z info cache Root cert has changed, start rotating root cert 2026-11-03T21:56:49.690282Z info cache returned workload trust anchor from cache ttl=23h59m59.309718731s 2026-11-03T21:56:49.709566Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T21:56:49.816562Z info ads ADS: new connection for node:1 2026-11-03T21:56:49.816655Z info cache returned workload certificate from cache ttl=23h59m59.183346343s 2026-11-03T21:56:49.816736Z info ads ADS: new connection for node:2 2026-11-03T21:56:49.816810Z info cache returned workload trust anchor from cache ttl=23h59m59.183191087s 2026-11-03T21:56:49.897893Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:56:57.866520Z info Readiness succeeded in 8.238255825s 2026-11-03T21:56:57.866965Z info Envoy proxy is ready 2026-11-03T21:57:06.810161Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T21:57:06.810203Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T21:57:06.810218Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-03T21:57:06.810222Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 [2026-11-03T21:57:06.694Z] "DELETE /v1/api-keys/ae94cc56-4c37-437d-acd6-6aeb8f82a59a HTTP/1.1" 200 - via_upstream - "-" 0 365 116 - "34.228.250.194,10.132.0.12" "curl/7.76.1" "8a23c361-3b62-4875-9433-8d32dd16382e" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "10.132.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.43:46214 10.134.0.43:443 10.132.0.12:34904 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:59:07.008323Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:59:07.008364Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:59:07.008378Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:59:07.008385Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T21:59:06.938Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 69 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "bb957609-ee28-40d6-ab01-18f949b4d723" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "10.132.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.43:56280 10.134.0.43:443 10.132.0.12:39884 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:59:07.190043Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T21:59:07.190076Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T21:59:07.190089Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T21:59:07.190094Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-03T21:59:08.020520Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:59:08.020555Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:59:08.020569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:59:08.020574Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T21:59:07.127Z] "DELETE /v1/api-keys/88b85706-fa90-4b02-81f1-b63f5c9376c8 HTTP/1.1" 200 - via_upstream - "-" 0 321 62 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "edab8e5a-dce8-44d1-b419-f7c3acf9aa1e" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "10.132.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.43:56284 10.134.0.43:443 10.132.0.12:39898 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T21:59:07.253Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 38 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "4ae2f283-36ed-4082-a6de-a34bf7feb2bd" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.134.0.43:443 10.132.0.12:39904 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T21:59:07.941Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 79 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "fd37d7b0-633c-4f25-ba1a-cacd542a5ced" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "10.132.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.43:56280 10.134.0.43:443 10.132.0.12:39918 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:59:08.356971Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T21:59:08.357004Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T21:59:08.357018Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-03T21:59:08.357023Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 [2026-11-03T21:59:08.258Z] "DELETE /v1/api-keys/4421ed6e-d988-49e4-a73f-f5e4e2606d68 HTTP/1.1" 200 - via_upstream - "-" 0 321 98 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "186ef3c0-0b74-46f9-b11b-b7cab6d0845f" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "10.132.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.43:56284 10.134.0.43:443 10.132.0.12:39924 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:59:14.054106Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:59:13.542Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 63 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "02873591-fcf0-48bc-9bcd-feca4cbebae7" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 10.132.0.12:59478 - v1-completions-path 2026-11-03T21:59:28.027698Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:00:26.163627Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:00:26.930438Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:00:26.930513Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:00:26.930540Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:00:26.930548Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T22:00:26.819Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 313 111 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "ac4e9c1e-5c26-4566-afa1-4d0255d69330" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "10.132.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.43:58280 10.134.0.43:443 10.132.0.12:42664 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T22:00:48.751388Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:01:25.287074Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T22:01:25.287099Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T22:01:25.287103Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T22:01:25.287106Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-11-03T22:01:25.254Z] "POST /llm/premium-simulated-simulated-premium/v1/completions HTTP/1.1" 403 - - "-" 74 77 33 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "c9412d9e-15d6-4a63-bc04-dd1b2e9092ca" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "-" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 10.132.0.12:53436 - v1-completions-path 2026-11-03T22:01:29.694259Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T22:01:33.493Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 72 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "a0280b74-26fb-4a90-9fa1-35c37aa76c57" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "10.134.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:37460 10.134.0.43:443 10.132.0.12:53202 - v1-completions-path 2026-11-03T22:01:45.491151Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:01:47.003600Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T22:01:52.443Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 57 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "be6b898b-35f6-4b3a-8e8d-d5cf99d77eab" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 10.132.0.12:44424 - v1-completions-path [2026-11-03T22:01:52.562Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 51 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "2b42b64b-bef4-4a8c-989b-57235ec4ba36" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 10.132.0.12:44440 - v1-completions-path 2026-11-03T22:02:04.130886Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:02:05.707588Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:02:08.461474Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:02:16.851893Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:02:24.373631Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-11-03T22:02:24.315Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 59 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "2c56e887-d7e0-4c29-a85f-2d652cf6b378" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "10.134.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:37460 10.134.0.43:443 10.132.0.12:56252 - v1-completions-path [2026-11-03T22:02:24.559Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 55 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "611de3de-35ba-4cdb-a032-da701ced84ce" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "10.134.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:60502 10.134.0.43:443 10.132.0.12:56262 - v1-completions-path [2026-11-03T22:02:24.884Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 56 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "46fd7255-ecdf-4c4c-8dc1-5b866625785c" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "10.134.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:60502 10.134.0.43:443 10.132.0.12:56278 - v1-completions-path [2026-11-03T22:02:25.113Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 55 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "0fcf99d9-d03e-444e-a0d6-95f56398f9c1" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "10.134.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:37460 10.134.0.43:443 10.132.0.12:56288 - v1-completions-path [2026-11-03T22:02:25.667Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 57 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "15b0209f-ca6b-4894-8479-5ca6cf5f76db" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "10.134.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:37460 10.134.0.43:443 10.132.0.12:56290 - v1-completions-path 2026-11-03T22:02:41.054983Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:02:51.382708Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:02:53.031314Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:02:58.697381Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:02:58.697419Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:02:58.697435Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T22:02:58.697440Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T22:02:58.636Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 317 60 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "6ad0e919-44c9-4836-898e-e0a44c973cc9" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "10.132.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.43:40740 10.134.0.43:443 10.132.0.12:54050 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T22:02:59.989139Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=26 [2026-11-03T22:02:59.569Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 56 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "eeb84377-2e20-4447-8143-4cafa4f529e3" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "10.134.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:37460 10.134.0.43:443 10.132.0.12:54062 - v1-completions-path [2026-11-03T22:02:59.938Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 55 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "34812c3b-4e39-4b15-8a20-de08314104bc" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "10.134.0.37:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:60502 10.134.0.43:443 10.132.0.12:54074 - v1-completions-path 2026-11-03T22:03:11.989011Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:03:13.610945Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:03:30.119093Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:03:32.125923Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:03:48.902275Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:03:50.347335Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:04:06.712995Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:04:17.414252Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:04:20.201544Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T22:04:34.484Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 34 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "8480c7c5-8a09-403e-a28d-7ef51ade0589" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 10.132.0.12:37284 - v1-completions-path 2026-11-03T22:04:35.469632Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:04:36.910587Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:04:41.880907Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T22:04:41.880943Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T22:04:41.880956Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T22:04:41.880961Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-03T22:04:41.737Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 31 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "c9d521f3-cc29-427f-a828-914f974b9787" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 10.132.0.12:37304 - v1-completions-path [2026-11-03T22:04:41.821Z] "DELETE /v1/api-keys/06c91851-f234-4fe7-983e-2e6f5f22eb0b HTTP/1.1" 404 - via_upstream - "-" 0 29 59 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "4a9c86fa-af50-405c-ac54-2af3850e058a" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "10.132.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.43:53432 10.134.0.43:443 10.132.0.12:37316 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T22:04:56.003633Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:04:57.581866Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:06:26.024163Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:06:27.772562Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:07:53.726728Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:08:08.492908Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:08:10.032507Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:08:12.689750Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:08:12.689789Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:08:12.689807Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T22:08:12.689812Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T22:08:12.589Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 65 307 100 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "0521a488-3777-46ea-ba22-79b73c70c991" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "10.132.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.43:46816 10.134.0.43:443 10.132.0.12:44834 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T22:08:24.481247Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:09:32.715841Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T22:09:32.715881Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T22:09:32.715899Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T22:09:32.715905Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-03T22:09:32.649Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 568 66 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "9fe7047e-d0ed-415f-aae1-44b457dd5b69" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "10.132.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.43:34782 10.134.0.43:443 10.132.0.12:44980 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T22:09:37.615136Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:09:54.173491Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:10:10.342991Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:10:22.012925Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:10:25.676380Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:10:37.010100Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:10:39.553915Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:10:57.400541Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:11:01.698887Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:11:04.441764Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:11:04.441802Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:11:04.441819Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T22:11:04.441824Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:11:04.599164Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=26 [2026-11-03T22:11:04.340Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 324 101 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "3c7fbf91-4055-437e-95c2-beb353ab70cb" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "10.132.0.57:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.43:51108 10.134.0.43:443 10.132.0.12:55994 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T22:11:04.514Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 84 - "34.228.250.194,10.132.0.12" "python-requests/2.34.2" "b4e6811c-f15e-42e6-b9f3-128111d87c40" "maas.apps.6a40f9c86944f74aa03d.prod.konflux-ocp-ci.dev" "10.134.0.36:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:51406 10.134.0.43:443 10.132.0.12:56010 - v1-catch-all-path 2026-11-03T22:11:06.836176Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:11:22.099939Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:11:23.520666Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:11:38.095956Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T22:11:48.860214Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8