2026-10-30T14:52:42.445948Z info FLAG: --concurrency="0" 2026-10-30T14:52:42.445967Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T14:52:42.445971Z info FLAG: --help="false" 2026-10-30T14:52:42.445973Z info FLAG: --log_as_json="false" 2026-10-30T14:52:42.445975Z info FLAG: --log_caller="" 2026-10-30T14:52:42.445977Z info FLAG: --log_output_level="default:info" 2026-10-30T14:52:42.445980Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T14:52:42.445984Z info FLAG: --log_target="[stdout]" 2026-10-30T14:52:42.445987Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T14:52:42.445989Z info FLAG: --outlierLogPath="" 2026-10-30T14:52:42.445991Z info FLAG: --profiling="true" 2026-10-30T14:52:42.445993Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T14:52:42.445995Z info FLAG: --proxyLogLevel="warning" 2026-10-30T14:52:42.445997Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T14:52:42.445999Z info FLAG: --stsPort="0" 2026-10-30T14:52:42.446001Z info FLAG: --templateFile="" 2026-10-30T14:52:42.446003Z info FLAG: --tokenManagerPlugin="" 2026-10-30T14:52:42.446006Z info FLAG: --vklog="0" 2026-10-30T14:52:42.446009Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T14:52:42.446153Z info Proxy role ips=[10.133.1.67] type=router id=maas-default-gateway-openshift-default-c9ffffc7f-fb779.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T14:52:42.446196Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T14:52:42.447292Z info cpu limit detected as 2, setting concurrency 2026-10-30T14:52:42.447843Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T14:52:42.447862Z info JWT policy is third-party-jwt 2026-10-30T14:52:42.447867Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T14:52:42.450195Z info Starting default Istio SDS Server 2026-10-30T14:52:42.450216Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T14:52:42.450181Z info Opening status port 15020 2026-10-30T14:52:42.450241Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T14:52:42.450941Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T14:52:42.452632Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T14:52:42.453750Z info Starting proxy agent 2026-10-30T14:52:42.453790Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T14:52:42.460432Z info sds Starting SDS grpc server 2026-10-30T14:52:42.460454Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T14:52:42.498512Z info cache generated new workload certificate resourceName=default latency=47.413323ms ttl=23h59m59.501493911s 2026-10-30T14:52:42.498586Z info cache Root cert has changed, start rotating root cert 2026-10-30T14:52:42.498628Z info cache returned workload trust anchor from cache ttl=23h59m59.501373566s 2026-10-30T14:52:42.521319Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T14:52:42.634017Z info ads ADS: new connection for node:1 2026-10-30T14:52:42.634116Z info cache returned workload certificate from cache ttl=23h59m59.365885091s 2026-10-30T14:52:42.634346Z info ads ADS: new connection for node:2 2026-10-30T14:52:42.634422Z info cache returned workload trust anchor from cache ttl=23h59m59.365579235s 2026-10-30T14:52:42.719673Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:52:51.505397Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:52:52.400185Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-10-30T14:52:53.084722Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-10-30T14:52:53.643176Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:52:55.079692Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-10-30T14:52:55.127063Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-10-30T14:52:56.075351Z info Readiness succeeded in 13.631593965s 2026-10-30T14:52:56.075679Z info Envoy proxy is ready 2026-10-30T14:53:08.409148Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:53:10.565232Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:53:13.685611Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:53:23.990832Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-30T14:53:39.211Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 77 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "84c42b5b-5ee8-4d58-a345-df98640f84b1" "maas.apps.1190c9c9fba3d97157e0.prod.konflux-ocp-ci.dev" "10.133.0.50:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.67:39376 10.133.1.67:443 10.132.0.19:47574 - v1-completions-path [2026-10-30T14:53:39.447Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 62 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "62fdafd2-5798-42b0-a5a6-b67c1e19ccfc" "maas.apps.1190c9c9fba3d97157e0.prod.konflux-ocp-ci.dev" "10.133.0.50:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.67:39392 10.133.1.67:443 10.132.0.19:47590 - v1-completions-path 2026-10-30T14:53:56.329039Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:54:06.700048Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:54:11.016468Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 [2026-10-30T14:54:25.399Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 61 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "e8740cb0-474b-4a3a-9749-b2125fa3cbaf" "maas.apps.1190c9c9fba3d97157e0.prod.konflux-ocp-ci.dev" "10.133.0.50:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.67:39376 10.133.1.67:443 10.132.0.19:33478 - v1-completions-path [2026-10-30T14:54:25.991Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 51 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "fdbcee3b-0c68-4dbc-bc3c-ca9ac438a5ad" "maas.apps.1190c9c9fba3d97157e0.prod.konflux-ocp-ci.dev" "10.133.0.50:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.1.67:39392 10.133.1.67:443 10.132.0.19:33480 - v1-completions-path 2026-10-30T14:54:38.467501Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:54:49.751825Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:55:06.908129Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:55:21.260882Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:55:38.127414Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:55:54.147088Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:56:15.026982Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:56:17.991700Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:56:37.131200Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T14:56:39.873595Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T15:00:07.190948Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T15:00:07.190983Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T15:00:07.190997Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T15:00:07.191002Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T15:00:07.110Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 80 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "74c47f15-0f08-4cb7-b8ea-26d8084780e6" "maas.apps.1190c9c9fba3d97157e0.prod.konflux-ocp-ci.dev" "10.132.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.1.67:48748 10.133.1.67:443 10.132.0.19:37578 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T15:00:26.699932Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T15:00:45.915000Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T15:00:45.915032Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T15:00:45.915044Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T15:00:45.915048Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T15:00:45.915054Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-10-30T15:00:45.843Z] "DELETE /v1/api-keys/e2e-trlp-test-key HTTP/1.1" 404 - via_upstream - "-" 0 29 71 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "795cf1d2-b464-4367-a02f-0eb7ddda9b35" "maas.apps.1190c9c9fba3d97157e0.prod.konflux-ocp-ci.dev" "10.132.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.1.67:48748 10.133.1.67:443 10.132.0.19:58356 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T15:00:57.952571Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T15:01:00.399551Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T15:01:13.062440Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T15:02:49.082679Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T15:02:49.082713Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T15:02:49.082728Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-30T15:02:49.082733Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-30T15:02:49.005Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 33 325 77 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "750a66bb-12d8-4071-beed-46a889ddc10f" "maas.apps.1190c9c9fba3d97157e0.prod.konflux-ocp-ci.dev" "10.132.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.1.67:56222 10.133.1.67:443 10.132.0.19:55018 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T15:02:53.738755Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T15:03:21.332211Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T15:03:44.710142Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T15:03:44.710182Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T15:03:44.710199Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T15:03:44.710204Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T15:03:44.654Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 630 55 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "b812e16b-18bf-468a-b8d6-e3c2551bdbef" "maas.apps.1190c9c9fba3d97157e0.prod.konflux-ocp-ci.dev" "10.132.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.1.67:51034 10.133.1.67:443 10.132.0.19:60574 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T15:03:46.892906Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T15:03:58.974530Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T15:04:03.299876Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T15:04:06.887944Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T15:04:17.266355Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T15:04:21.773480Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T15:04:39.895000Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T15:04:57.658658Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T15:04:57.658686Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T15:04:57.658689Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-30T15:04:57.658694Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-10-30T15:04:57.610Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 401 - - "-" 61 88 48 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "75e3c528-7649-43c6-88d2-f7aa1a369c52" "maas.apps.1190c9c9fba3d97157e0.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.1.67:443 10.132.0.19:47828 - v1-catch-all-path 2026-10-30T15:05:50.919783Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T15:06:18.347564Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T15:06:20.609927Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T15:06:40.814339Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T15:07:02.351117Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-10-30T15:07:19.667070Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9