2026-11-03T07:49:45.135989Z info FLAG: --concurrency="0" 2026-11-03T07:49:45.136030Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T07:49:45.136036Z info FLAG: --help="false" 2026-11-03T07:49:45.136039Z info FLAG: --log_as_json="false" 2026-11-03T07:49:45.136042Z info FLAG: --log_caller="" 2026-11-03T07:49:45.136044Z info FLAG: --log_output_level="default:info" 2026-11-03T07:49:45.136048Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T07:49:45.136054Z info FLAG: --log_target="[stdout]" 2026-11-03T07:49:45.136057Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T07:49:45.136060Z info FLAG: --outlierLogPath="" 2026-11-03T07:49:45.136063Z info FLAG: --profiling="true" 2026-11-03T07:49:45.136066Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T07:49:45.136069Z info FLAG: --proxyLogLevel="warning" 2026-11-03T07:49:45.136072Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T07:49:45.136075Z info FLAG: --stsPort="0" 2026-11-03T07:49:45.136078Z info FLAG: --templateFile="" 2026-11-03T07:49:45.136081Z info FLAG: --tokenManagerPlugin="" 2026-11-03T07:49:45.136085Z info FLAG: --vklog="0" 2026-11-03T07:49:45.136089Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T07:49:45.136258Z info Proxy role ips=[10.132.0.126] type=router id=maas-default-gateway-openshift-default-58d898bd55-xqv52.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T07:49:45.136323Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T07:49:45.138003Z info cpu limit detected as 2, setting concurrency 2026-11-03T07:49:45.138322Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T07:49:45.138336Z info JWT policy is third-party-jwt 2026-11-03T07:49:45.138339Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T07:49:45.139778Z info Starting default Istio SDS Server 2026-11-03T07:49:45.139808Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T07:49:45.139840Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T07:49:45.140390Z info Opening status port 15020 2026-11-03T07:49:45.141684Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T07:49:45.142856Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T07:49:45.143606Z info Starting proxy agent 2026-11-03T07:49:45.143635Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T07:49:45.151957Z info sds Starting SDS grpc server 2026-11-03T07:49:45.151983Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T07:49:45.211098Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T07:49:45.244373Z info cache generated new workload certificate resourceName=default latency=102.481785ms ttl=23h59m59.755630955s 2026-11-03T07:49:45.244417Z info cache Root cert has changed, start rotating root cert 2026-11-03T07:49:45.244453Z info cache returned workload trust anchor from cache ttl=23h59m59.755547512s 2026-11-03T07:49:45.376215Z info ads ADS: new connection for node:1 2026-11-03T07:49:45.376273Z info cache returned workload certificate from cache ttl=23h59m59.623728193s 2026-11-03T07:49:45.376639Z info ads ADS: new connection for node:2 2026-11-03T07:49:45.376708Z info cache returned workload trust anchor from cache ttl=23h59m59.623292782s 2026-11-03T07:49:45.461619Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T07:49:53.626207Z info Readiness succeeded in 8.492754264s 2026-11-03T07:49:53.626520Z info Envoy proxy is ready 2026-11-03T07:49:55.777395Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T07:50:39.257263Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T07:50:39.257299Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T07:50:39.257311Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-03T07:50:39.257316Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 [2026-11-03T07:50:39.138Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 74 316 118 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "33b0538d-89c9-4381-ae84-db4d1f5e3eab" "maas.apps.5db8d7cd7d1f531c64b5.prod.konflux-ocp-ci.dev" "10.133.0.35:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.126:60564 10.132.0.126:443 10.132.0.20:50348 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T07:50:49.400454Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T07:50:55.692Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 79 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "d28a09d6-7e94-4e03-87c2-e5b1ea81f460" "maas.apps.5db8d7cd7d1f531c64b5.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.126:36858 10.132.0.126:443 10.132.0.20:44970 - v1-completions-path 2026-11-03T07:51:10.917155Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T07:51:15.687868Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T07:51:27.627774Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T07:51:27.627810Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T07:51:27.627823Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T07:51:27.627827Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T07:51:27.508Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 336 119 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "710d2988-faab-4602-87c2-06a6fb5b7eb2" "maas.apps.5db8d7cd7d1f531c64b5.prod.konflux-ocp-ci.dev" "10.133.0.35:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.126:53732 10.132.0.126:443 10.132.0.20:57506 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T07:51:42.317127Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T07:51:59.508249Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T07:52:30.535550Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T07:52:41.295857Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T07:52:47.794862Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T07:53:00.438Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 69 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "4ec87577-254b-492f-8006-974b1c7c0813" "maas.apps.5db8d7cd7d1f531c64b5.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.126:56434 10.132.0.126:443 10.132.0.20:52144 - v1-completions-path [2026-11-03T07:53:00.970Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 65 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "7f60153d-13ff-4c06-81db-467422cf04c4" "maas.apps.5db8d7cd7d1f531c64b5.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.126:36858 10.132.0.126:443 10.132.0.20:52154 - v1-completions-path 2026-11-03T07:53:16.225953Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T07:53:21.127577Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T07:53:44.357194Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T07:53:50.902730Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T07:54:16.912084Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T07:54:38.843225Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T07:54:52.144419Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T07:54:58.116677Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T07:55:25.803428Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T07:55:31.427253Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T07:55:54.513136Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T07:56:02.900929Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T07:56:02.900969Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T07:56:02.900982Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T07:56:02.900987Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-03T07:56:02.790Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 67 309 110 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "e1e31d87-a3ce-4b42-af46-110d43ae5d49" "maas.apps.5db8d7cd7d1f531c64b5.prod.konflux-ocp-ci.dev" "10.133.0.35:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.126:49336 10.132.0.126:443 10.132.0.20:51320 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T07:56:17.618203Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T07:56:28.367878Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T07:56:44.850Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 36 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "1b1f7ccc-40d2-4618-a039-59335d072eaa" "maas.apps.5db8d7cd7d1f531c64b5.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.126:443 10.132.0.20:53528 - v1-completions-path 2026-11-03T07:56:49.063966Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T07:56:55.169934Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T07:59:16.506704Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T07:59:22.601114Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T08:01:00.716665Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T08:01:18.037188Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T08:01:22.700938Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T08:01:47.028608Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T08:03:58.469438Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T08:04:44.207068Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T08:05:12.042114Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T08:05:27.596138Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T08:05:46.374599Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T08:05:58.938214Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T08:06:03.896130Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T08:06:17.454219Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=26 [2026-11-03T08:06:17.395Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 63 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "1a06faf7-98dd-4dc8-841d-5f370d7fd2e5" "maas.apps.5db8d7cd7d1f531c64b5.prod.konflux-ocp-ci.dev" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.126:36858 10.132.0.126:443 10.132.0.20:55696 - v1-completions-path 2026-11-03T08:06:31.942635Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T08:10:09.662625Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T08:10:22.624060Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T08:10:29.160474Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-11-03T08:10:29.161886Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-11-03T08:10:29.084Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 77 - "34.228.250.194,10.132.0.20" "python-requests/2.34.2" "6902ba69-a61d-41ea-b380-675810d6bbae" "maas.apps.5db8d7cd7d1f531c64b5.prod.konflux-ocp-ci.dev" "10.134.0.29:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.126:34280 10.132.0.126:443 10.132.0.20:36974 - v1-catch-all-path 2026-11-03T08:10:43.724158Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T08:10:48.422030Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T08:11:04.410940Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T08:11:16.437316Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8