2026-10-30T22:42:18.746617Z info FLAG: --concurrency="0" 2026-10-30T22:42:18.746634Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T22:42:18.746639Z info FLAG: --help="false" 2026-10-30T22:42:18.746642Z info FLAG: --log_as_json="false" 2026-10-30T22:42:18.746644Z info FLAG: --log_caller="" 2026-10-30T22:42:18.746646Z info FLAG: --log_output_level="default:info" 2026-10-30T22:42:18.746648Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T22:42:18.746653Z info FLAG: --log_target="[stdout]" 2026-10-30T22:42:18.746655Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T22:42:18.746657Z info FLAG: --outlierLogPath="" 2026-10-30T22:42:18.746659Z info FLAG: --profiling="true" 2026-10-30T22:42:18.746662Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T22:42:18.746665Z info FLAG: --proxyLogLevel="warning" 2026-10-30T22:42:18.746669Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T22:42:18.746672Z info FLAG: --stsPort="0" 2026-10-30T22:42:18.746675Z info FLAG: --templateFile="" 2026-10-30T22:42:18.746678Z info FLAG: --tokenManagerPlugin="" 2026-10-30T22:42:18.746681Z info FLAG: --vklog="0" 2026-10-30T22:42:18.746684Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T22:42:18.746828Z info Proxy role ips=[10.134.0.243] type=router id=maas-default-gateway-openshift-default-76597d6855-z8pdj.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T22:42:18.746874Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T22:42:18.748683Z info cpu limit detected as 2, setting concurrency 2026-10-30T22:42:18.748902Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T22:42:18.748914Z info JWT policy is third-party-jwt 2026-10-30T22:42:18.748918Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T22:42:18.752027Z info Starting default Istio SDS Server 2026-10-30T22:42:18.752099Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T22:42:18.752128Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T22:42:18.752049Z info Opening status port 15020 2026-10-30T22:42:18.752731Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T22:42:18.752902Z info sds Starting SDS grpc server 2026-10-30T22:42:18.752962Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T22:42:18.753988Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T22:42:18.754817Z info Starting proxy agent 2026-10-30T22:42:18.754851Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T22:42:18.822801Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T22:42:18.991099Z info ads ADS: new connection for node:1 2026-10-30T22:42:18.993619Z info ads ADS: new connection for node:2 2026-10-30T22:42:19.135045Z info cache generated new workload certificate resourceName=default latency=381.993586ms ttl=23h59m59.864957732s 2026-10-30T22:42:19.135083Z info cache Root cert has changed, start rotating root cert 2026-10-30T22:42:19.135115Z info cache returned workload certificate from cache ttl=23h59m59.864890057s 2026-10-30T22:42:19.135127Z info cache returned workload trust anchor from cache ttl=23h59m59.864873366s 2026-10-30T22:42:19.135210Z info cache returned workload trust anchor from cache ttl=23h59m59.864790386s 2026-10-30T22:42:19.135518Z info cache returned workload trust anchor from cache ttl=23h59m59.864483175s 2026-10-30T22:42:19.242424Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:42:27.205199Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:42:28.246794Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-10-30T22:42:29.399200Z info Readiness succeeded in 10.656919976s 2026-10-30T22:42:29.399736Z info Envoy proxy is ready 2026-10-30T22:43:18.669645Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-30T22:43:25.979Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 84 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "b491837d-ec8c-4e80-9316-463b1e8b9a29" "maas.apps.5f5985d19b2baf18959c.prod.konflux-ocp-ci.dev" "10.132.0.56:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.243:40858 10.134.0.243:443 10.132.0.20:58980 - v1-completions-path [2026-10-30T22:43:26.363Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 56 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "c124155c-5955-48da-ba63-3eb6e4864a62" "maas.apps.5f5985d19b2baf18959c.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.243:443 10.132.0.20:58994 - v1-completions-path 2026-10-30T22:43:40.021453Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:43:44.994687Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-30T22:43:57.480Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 68 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "8e49d0e6-8ff9-45f6-9099-4aa38d4986d7" "maas.apps.5f5985d19b2baf18959c.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.243:443 10.132.0.20:35344 - v1-completions-path 2026-10-30T22:44:15.037951Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:44:28.333257Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-30T22:44:38.375Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 66 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "f759e4a2-3f87-4162-acbd-3c73dd0ee153" "maas.apps.5f5985d19b2baf18959c.prod.konflux-ocp-ci.dev" "10.132.0.56:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.243:32892 10.134.0.243:443 10.132.0.20:49944 - v1-completions-path [2026-10-30T22:44:41.274Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 56 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "40a0134f-85d7-43da-9793-40cb547ec24f" "maas.apps.5f5985d19b2baf18959c.prod.konflux-ocp-ci.dev" "10.132.0.56:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.243:40858 10.134.0.243:443 10.132.0.20:49960 - v1-completions-path 2026-10-30T22:44:59.474216Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:45:10.312959Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:45:16.799766Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:45:31.998917Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-10-30T22:45:31.943Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 60 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "cceb30fe-c8c6-492a-ba12-0d9369710741" "maas.apps.5f5985d19b2baf18959c.prod.konflux-ocp-ci.dev" "10.132.0.56:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.243:40858 10.134.0.243:443 10.132.0.20:59436 - v1-completions-path [2026-10-30T22:45:34.261Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 55 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "5bf971aa-5305-4934-aad8-6027187e904e" "maas.apps.5f5985d19b2baf18959c.prod.konflux-ocp-ci.dev" "10.132.0.56:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.243:32892 10.134.0.243:443 10.132.0.20:54250 - v1-completions-path 2026-10-30T22:45:48.495964Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:45:53.144990Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:46:05.883733Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T22:46:05.883770Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T22:46:05.883784Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T22:46:05.883790Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T22:46:05.767Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 73 316 116 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "e5a11964-8d6b-430f-affe-fb6785c8f115" "maas.apps.5f5985d19b2baf18959c.prod.konflux-ocp-ci.dev" "10.133.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.243:49188 10.134.0.243:443 10.132.0.20:33184 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T22:46:19.787550Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:46:26.431506Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:46:52.379444Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:47:14.671886Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:47:27.823688Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:47:34.412696Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:48:01.874720Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:48:07.635875Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-30T22:48:19.315Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 76 0 35 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "240ed3bc-4e68-4d5d-a4bf-a4ef5307750a" "maas.apps.5f5985d19b2baf18959c.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.243:443 10.132.0.20:37532 - v1-completions-path 2026-10-30T22:48:27.723041Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:48:47.739172Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:48:58.716174Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:49:18.016766Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:49:24.034020Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:53:25.807537Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:53:43.393236Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:53:48.088360Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:54:12.680700Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:56:25.737393Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:57:08.240464Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:57:39.050757Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:57:54.975246Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:58:12.279067Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:58:24.676204Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:58:29.850894Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T22:58:43.201364Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-10-30T22:58:43.141Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 64 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "bfa82aee-10dc-465b-ab6d-8eb2e30bb3ef" "maas.apps.5f5985d19b2baf18959c.prod.konflux-ocp-ci.dev" "10.132.0.56:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.243:32892 10.134.0.243:443 10.132.0.20:53978 - v1-completions-path 2026-10-30T22:58:57.555325Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T23:02:33.869322Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T23:02:39.673459Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-10-30T23:02:39.593Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 79 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "f2b04fdc-27c9-4611-b455-7e70aa75b658" "maas.apps.5f5985d19b2baf18959c.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.243:48760 10.134.0.243:443 10.132.0.20:56952 - v1-catch-all-path 2026-10-30T23:02:45.339767Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T23:02:53.826256Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T23:02:53.826294Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T23:02:53.826309Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T23:02:53.826313Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T23:02:53.705Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 317 121 - "52.71.36.72,10.132.0.20" "python-requests/2.34.2" "e30d94e1-0e6c-492f-85be-38f36cbd026e" "maas.apps.5f5985d19b2baf18959c.prod.konflux-ocp-ci.dev" "10.133.0.108:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.243:42704 10.134.0.243:443 10.132.0.20:56606 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T23:03:08.791508Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T23:03:13.439656Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T23:03:29.999343Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T23:03:41.606084Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9