2026-10-30T06:25:26.296482Z info FLAG: --concurrency="0" 2026-10-30T06:25:26.296500Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T06:25:26.296505Z info FLAG: --help="false" 2026-10-30T06:25:26.296507Z info FLAG: --log_as_json="false" 2026-10-30T06:25:26.296509Z info FLAG: --log_caller="" 2026-10-30T06:25:26.296511Z info FLAG: --log_output_level="default:info" 2026-10-30T06:25:26.296513Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T06:25:26.296517Z info FLAG: --log_target="[stdout]" 2026-10-30T06:25:26.296519Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T06:25:26.296521Z info FLAG: --outlierLogPath="" 2026-10-30T06:25:26.296523Z info FLAG: --profiling="true" 2026-10-30T06:25:26.296525Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T06:25:26.296527Z info FLAG: --proxyLogLevel="warning" 2026-10-30T06:25:26.296529Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T06:25:26.296531Z info FLAG: --stsPort="0" 2026-10-30T06:25:26.296533Z info FLAG: --templateFile="" 2026-10-30T06:25:26.296534Z info FLAG: --tokenManagerPlugin="" 2026-10-30T06:25:26.296537Z info FLAG: --vklog="0" 2026-10-30T06:25:26.296539Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T06:25:26.296675Z info Proxy role ips=[10.133.1.30] type=router id=maas-default-gateway-openshift-default-84587756dc-x56kg.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T06:25:26.296715Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T06:25:26.297780Z info cpu limit detected as 2, setting concurrency 2026-10-30T06:25:26.298366Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T06:25:26.298379Z info JWT policy is third-party-jwt 2026-10-30T06:25:26.298382Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T06:25:26.299668Z info Starting default Istio SDS Server 2026-10-30T06:25:26.299691Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T06:25:26.299714Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T06:25:26.299961Z info Opening status port 15020 2026-10-30T06:25:26.300967Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T06:25:26.301301Z info sds Starting SDS grpc server 2026-10-30T06:25:26.301332Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T06:25:26.302110Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T06:25:26.302900Z info Starting proxy agent 2026-10-30T06:25:26.302935Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T06:25:26.366581Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T06:25:26.410618Z info cache generated new workload certificate resourceName=default latency=109.209454ms ttl=23h59m59.589388232s 2026-10-30T06:25:26.410670Z info cache Root cert has changed, start rotating root cert 2026-10-30T06:25:26.410714Z info cache returned workload trust anchor from cache ttl=23h59m59.58928677s 2026-10-30T06:25:26.479515Z info ads ADS: new connection for node:1 2026-10-30T06:25:26.479631Z info cache returned workload certificate from cache ttl=23h59m59.520371245s 2026-10-30T06:25:26.479884Z info ads ADS: new connection for node:2 2026-10-30T06:25:26.479957Z info cache returned workload trust anchor from cache ttl=23h59m59.520044195s 2026-10-30T06:25:26.564153Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:25:34.398279Z info Readiness succeeded in 8.104593542s 2026-10-30T06:25:34.398704Z info Envoy proxy is ready 2026-10-30T06:25:56.792325Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T06:26:01.755Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 85 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "10e0e4c4-8e45-461c-b922-43179e3b2508" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.30:50782 10.133.1.30:443 10.132.0.20:56612 - v1-completions-path 2026-10-30T06:26:13.914092Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:26:17.937567Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:26:33.961977Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:26:36.359042Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:26:39.272927Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:26:49.622562Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T06:26:55.419Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 75 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "50b1d751-1c2b-45d1-8413-0d912dd89c49" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.30:41808 10.133.1.30:443 10.132.0.20:57226 - v1-completions-path [2026-10-30T06:26:55.939Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 56 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "a266e9e5-6e3d-4810-ad24-3d60716ec632" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.30:50782 10.133.1.30:443 10.132.0.20:46362 - v1-completions-path 2026-10-30T06:27:12.645416Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:27:23.112221Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:27:27.234388Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T06:27:31.474Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 299 61 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "3615e969-ce68-482d-b8b4-07e91e114461" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.30:41808 10.133.1.30:443 10.132.0.20:50702 - v1-completions-path [2026-10-30T06:27:31.921Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 59 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "d3e4ae2e-9bb2-4214-8dea-719340d0a531" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.30:50782 10.133.1.30:443 10.132.0.20:50714 - v1-completions-path [2026-10-30T06:27:32.265Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 56 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "62a135c4-f831-4737-9024-30f061fb18d2" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.30:41808 10.133.1.30:443 10.132.0.20:50728 - v1-completions-path 2026-10-30T06:27:32.648413Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=26 [2026-10-30T06:27:32.598Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 54 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "5aabe1f1-3327-4cf4-bcfb-e754850634c1" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.30:41808 10.133.1.30:443 10.132.0.20:50740 - v1-completions-path 2026-10-30T06:27:44.685749Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:27:46.763727Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:28:06.217950Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:28:10.662389Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:28:26.281712Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:28:28.400634Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:28:46.435317Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:28:57.331968Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:29:00.471199Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:29:05.636267Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-10-30T06:29:05.646706Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:29:05.646740Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:29:05.646754Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:29:05.646760Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T06:29:05.570Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 323 76 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "06176b44-ed1e-4e25-b875-1ae21fb58e65" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.1.30:35450 10.133.1.30:443 10.132.0.20:36928 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T06:29:16.286Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 36 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "a13f49a2-8a8e-4dba-9cd0-aea9a8716502" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.1.30:443 10.132.0.20:54064 - v1-completions-path 2026-10-30T06:29:18.306083Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:29:20.843142Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:29:41.913957Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:29:44.599680Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:31:20.654409Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:31:23.998584Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:32:35.156592Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:32:35.156634Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T06:32:35.156649Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T06:32:35.156653Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T06:32:35.080Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 313 75 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "2539d6a6-faee-44a7-9719-5214fab19b9e" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.1.30:40232 10.133.1.30:443 10.132.0.20:54900 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T06:32:54.417291Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:33:09.972635Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:33:12.046025Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:33:27.062152Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:34:44.966479Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:35:00.374517Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:35:16.382269Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:35:30.600926Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:35:33.650722Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:35:46.386846Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:35:49.985422Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:35:57.565167Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T06:35:57.565229Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-10-30T06:35:57.565245Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-10-30T06:35:57.565249Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 [2026-10-30T06:35:57.491Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 336 73 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "cb87dcbb-4214-4073-8684-135552cd3842" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.1.30:41874 10.133.1.30:443 10.132.0.20:37686 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T06:35:57.745Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 59 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "5a7ed93a-0ea8-4281-bb18-8f735870abb1" "maas.apps.3b714319aaccfb1f8a93.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.30:41808 10.133.1.30:443 10.132.0.20:37706 - v1-completions-path 2026-10-30T06:36:09.821686Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:36:19.544239Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:36:27.678869Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:36:41.634659Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:36:43.688444Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:36:58.452914Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T06:37:09.500259Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8