2026-11-01T23:15:25.757096Z info FLAG: --concurrency="0" 2026-11-01T23:15:25.757113Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-01T23:15:25.757117Z info FLAG: --help="false" 2026-11-01T23:15:25.757120Z info FLAG: --log_as_json="false" 2026-11-01T23:15:25.757122Z info FLAG: --log_caller="" 2026-11-01T23:15:25.757124Z info FLAG: --log_output_level="default:info" 2026-11-01T23:15:25.757126Z info FLAG: --log_stacktrace_level="default:none" 2026-11-01T23:15:25.757135Z info FLAG: --log_target="[stdout]" 2026-11-01T23:15:25.757137Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-01T23:15:25.757139Z info FLAG: --outlierLogPath="" 2026-11-01T23:15:25.757141Z info FLAG: --profiling="true" 2026-11-01T23:15:25.757143Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-01T23:15:25.757145Z info FLAG: --proxyLogLevel="warning" 2026-11-01T23:15:25.757147Z info FLAG: --serviceCluster="istio-proxy" 2026-11-01T23:15:25.757149Z info FLAG: --stsPort="0" 2026-11-01T23:15:25.757151Z info FLAG: --templateFile="" 2026-11-01T23:15:25.757152Z info FLAG: --tokenManagerPlugin="" 2026-11-01T23:15:25.757161Z info FLAG: --vklog="0" 2026-11-01T23:15:25.757163Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-01T23:15:25.757297Z info Proxy role ips=[10.134.0.189] type=router id=maas-default-gateway-openshift-default-7f694df8bc-zvlcq.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-01T23:15:25.757342Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-01T23:15:25.758785Z info cpu limit detected as 2, setting concurrency 2026-11-01T23:15:25.758993Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-01T23:15:25.759004Z info JWT policy is third-party-jwt 2026-11-01T23:15:25.759007Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-01T23:15:25.760578Z info Starting default Istio SDS Server 2026-11-01T23:15:25.760597Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-01T23:15:25.760614Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-01T23:15:25.760946Z info Opening status port 15020 2026-11-01T23:15:25.761840Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-01T23:15:25.762580Z info sds Starting SDS grpc server 2026-11-01T23:15:25.762617Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-01T23:15:25.763149Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-01T23:15:25.763978Z info Starting proxy agent 2026-11-01T23:15:25.764009Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-01T23:15:25.827010Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-01T23:15:25.942236Z info cache generated new workload certificate resourceName=default latency=179.511289ms ttl=23h59m59.057768173s 2026-11-01T23:15:25.942284Z info cache Root cert has changed, start rotating root cert 2026-11-01T23:15:25.942319Z info cache returned workload trust anchor from cache ttl=23h59m59.05768313s 2026-11-01T23:15:25.980228Z info ads ADS: new connection for node:1 2026-11-01T23:15:25.980287Z info cache returned workload certificate from cache ttl=23h59m59.019714222s 2026-11-01T23:15:25.980945Z info ads ADS: new connection for node:2 2026-11-01T23:15:25.981007Z info cache returned workload trust anchor from cache ttl=23h59m59.018994143s 2026-11-01T23:15:26.062583Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:15:31.863975Z error failed scraping envoy metrics: error scraping http://localhost:15090/stats/prometheus: Get "http://localhost:15090/stats/prometheus": dial tcp [::1]:15090: connect: connection refused 2026-11-01T23:15:34.027340Z info Readiness succeeded in 8.272642055s 2026-11-01T23:15:34.027663Z info Envoy proxy is ready 2026-11-01T23:15:38.246057Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:16:24.427770Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T23:16:24.427809Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T23:16:24.427835Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T23:16:24.427840Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-01T23:16:24.300Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 74 316 127 - "52.71.36.72,10.134.0.21" "python-requests/2.34.2" "ce92e20a-57f4-44bf-8e3c-7e1bd7580ae1" "maas.apps.02a4ba9dcb097e09f654.prod.konflux-ocp-ci.dev" "10.132.0.25:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.189:45850 10.134.0.189:443 10.134.0.21:59292 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T23:16:34.417716Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T23:16:41.393Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 74 - "52.71.36.72,10.134.0.21" "python-requests/2.34.2" "8c3e6819-f5b8-40fe-bc30-302141b8ccef" "maas.apps.02a4ba9dcb097e09f654.prod.konflux-ocp-ci.dev" "10.132.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.189:50954 10.134.0.189:443 10.134.0.21:60524 - v1-completions-path [2026-11-01T23:16:42.193Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 59 - "52.71.36.72,10.134.0.21" "python-requests/2.34.2" "f1da67be-a960-4322-9883-6de5c2fcb65b" "maas.apps.02a4ba9dcb097e09f654.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.189:443 10.134.0.21:60530 - v1-completions-path 2026-11-01T23:16:55.700169Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:17:01.633280Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:17:27.158178Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:17:44.029365Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T23:17:51.352Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 299 64 - "52.71.36.72,10.134.0.21" "python-requests/2.34.2" "9e624bf3-b2f7-4449-90ff-7e8d8d089678" "maas.apps.02a4ba9dcb097e09f654.prod.konflux-ocp-ci.dev" "10.132.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.189:50954 10.134.0.189:443 10.134.0.21:40956 - v1-completions-path [2026-11-01T23:17:51.994Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 58 - "52.71.36.72,10.134.0.21" "python-requests/2.34.2" "6ee7202d-724c-4744-b2d3-754a4182aecd" "maas.apps.02a4ba9dcb097e09f654.prod.konflux-ocp-ci.dev" "10.132.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.189:42260 10.134.0.189:443 10.134.0.21:40962 - v1-completions-path [2026-11-01T23:17:54.173Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 55 - "52.71.36.72,10.134.0.21" "python-requests/2.34.2" "a5b016d8-168c-402c-b66c-422116866dca" "maas.apps.02a4ba9dcb097e09f654.prod.konflux-ocp-ci.dev" "10.132.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.189:42260 10.134.0.189:443 10.134.0.21:59684 - v1-completions-path 2026-11-01T23:17:57.083595Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-11-01T23:17:57.025Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 60 - "52.71.36.72,10.134.0.21" "python-requests/2.34.2" "213ed307-2ea9-4c54-9fc9-97951641b93b" "maas.apps.02a4ba9dcb097e09f654.prod.konflux-ocp-ci.dev" "10.132.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.189:50954 10.134.0.189:443 10.134.0.21:59698 - v1-completions-path 2026-11-01T23:18:15.006052Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:18:25.456719Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:18:31.588645Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T23:18:44.031Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 62 - "52.71.36.72,10.134.0.21" "python-requests/2.34.2" "1e426af0-5964-455d-9800-f124449c7e4d" "maas.apps.02a4ba9dcb097e09f654.prod.konflux-ocp-ci.dev" "10.132.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.189:50954 10.134.0.189:443 10.134.0.21:57426 - v1-completions-path 2026-11-01T23:19:00.431829Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:19:05.031373Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:19:28.540722Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:19:34.577491Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:20:00.803102Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:20:22.931407Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:21:05.581100Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:21:26.979409Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:21:37.720621Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-01T23:21:52.122Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 34 - "52.71.36.72,10.134.0.21" "python-requests/2.34.2" "2c77d6f7-1a49-4e6a-8c40-a163c544201e" "maas.apps.02a4ba9dcb097e09f654.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.189:443 10.134.0.21:60928 - v1-completions-path 2026-11-01T23:25:44.966497Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:26:01.888576Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:26:06.315800Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:26:31.193082Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:28:37.511407Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:29:21.082740Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:29:51.145558Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:30:06.296566Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:30:12.583075Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:30:25.663547Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:30:30.313458Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:30:38.773845Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T23:30:38.773882Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-01T23:30:38.773899Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-01T23:30:38.773904Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-01T23:30:38.645Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 336 128 - "52.71.36.72,10.134.0.21" "python-requests/2.34.2" "ac163693-7ac9-4360-9280-19fc6f55aaca" "maas.apps.02a4ba9dcb097e09f654.prod.konflux-ocp-ci.dev" "10.132.0.25:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.189:43174 10.134.0.189:443 10.134.0.21:55826 - odh-ai-gateway-infra.maas-api-route.3 2026-11-01T23:30:53.735229Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:34:29.238007Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:34:44.449471Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:35:05.437265Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:35:09.711396Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:35:26.022430Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-01T23:35:37.666137Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8