2026-11-04T05:10:39.351377Z info FLAG: --concurrency="0" 2026-11-04T05:10:39.351399Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-04T05:10:39.351404Z info FLAG: --help="false" 2026-11-04T05:10:39.351407Z info FLAG: --log_as_json="false" 2026-11-04T05:10:39.351409Z info FLAG: --log_caller="" 2026-11-04T05:10:39.351411Z info FLAG: --log_output_level="default:info" 2026-11-04T05:10:39.351413Z info FLAG: --log_stacktrace_level="default:none" 2026-11-04T05:10:39.351418Z info FLAG: --log_target="[stdout]" 2026-11-04T05:10:39.351420Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-04T05:10:39.351422Z info FLAG: --outlierLogPath="" 2026-11-04T05:10:39.351425Z info FLAG: --profiling="true" 2026-11-04T05:10:39.351427Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-04T05:10:39.351430Z info FLAG: --proxyLogLevel="warning" 2026-11-04T05:10:39.351432Z info FLAG: --serviceCluster="istio-proxy" 2026-11-04T05:10:39.351435Z info FLAG: --stsPort="0" 2026-11-04T05:10:39.351437Z info FLAG: --templateFile="" 2026-11-04T05:10:39.351439Z info FLAG: --tokenManagerPlugin="" 2026-11-04T05:10:39.351442Z info FLAG: --vklog="0" 2026-11-04T05:10:39.351444Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-04T05:10:39.351622Z info Proxy role ips=[10.132.0.58] type=router id=maas-default-gateway-openshift-default-5d84bc6b95-8fqq2.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-04T05:10:39.351755Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-04T05:10:39.352994Z info cpu limit detected as 2, setting concurrency 2026-11-04T05:10:39.353613Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-04T05:10:39.353627Z info JWT policy is third-party-jwt 2026-11-04T05:10:39.353630Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-04T05:10:39.355695Z info Starting default Istio SDS Server 2026-11-04T05:10:39.355730Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-04T05:10:39.355757Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-04T05:10:39.356605Z info Opening status port 15020 2026-11-04T05:10:39.357091Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-04T05:10:39.358801Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-04T05:10:39.359810Z info Starting proxy agent 2026-11-04T05:10:39.359849Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-04T05:10:39.360451Z info sds Starting SDS grpc server 2026-11-04T05:10:39.360466Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-04T05:10:39.428177Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-04T05:10:39.457304Z info cache generated new workload certificate resourceName=default latency=100.347385ms ttl=23h59m59.542702634s 2026-11-04T05:10:39.457351Z info cache Root cert has changed, start rotating root cert 2026-11-04T05:10:39.457398Z info cache returned workload trust anchor from cache ttl=23h59m59.542602844s 2026-11-04T05:10:39.559438Z info ads ADS: new connection for node:1 2026-11-04T05:10:39.559742Z info cache returned workload certificate from cache ttl=23h59m59.44025983s 2026-11-04T05:10:39.560355Z info ads ADS: new connection for node:2 2026-11-04T05:10:39.560652Z info cache returned workload trust anchor from cache ttl=23h59m59.439349803s 2026-11-04T05:10:39.650232Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T05:10:48.141225Z info Readiness succeeded in 8.793192015s 2026-11-04T05:10:48.141680Z info Envoy proxy is ready 2026-11-04T05:13:06.387040Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T05:13:06.387078Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T05:13:06.387090Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T05:13:06.387096Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T05:13:06.299Z] "DELETE /v1/api-keys/27851b85-4452-470b-a5ff-7a864314a093 HTTP/1.1" 200 - via_upstream - "-" 0 321 87 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "9af24646-3632-4007-bae7-ad691831851e" "maas.apps.0a41f24c8cfe96cff5be.prod.konflux-ocp-ci.dev" "10.134.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.58:36948 10.132.0.58:443 10.132.0.13:52242 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T05:13:07.411Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 55 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "81aea04a-e32c-470b-8ddb-ecaf108e5b89" "maas.apps.0a41f24c8cfe96cff5be.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.132.0.58:443 10.132.0.13:52250 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T05:13:16.995Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 281 119 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "947596a4-b2ec-4088-b040-eb8b867b1450" "maas.apps.0a41f24c8cfe96cff5be.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.58:59088 10.132.0.58:443 10.132.0.13:38992 - v1-completions-path [2026-11-04T05:13:17.871Z] "POST /maas-api/internal/v1/api-keys/cleanup HTTP/1.1" 404 NR route_not_found - "-" 2 0 11 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "a5955113-7186-4577-aac8-17cea47f1003" "maas.apps.0a41f24c8cfe96cff5be.prod.konflux-ocp-ci.dev" "-" - - 10.132.0.58:443 10.132.0.13:38996 - - [2026-11-04T05:13:20.590Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 135 2059 74 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "fa088c37-f70a-405f-9a4b-982439381c45" "maas.apps.0a41f24c8cfe96cff5be.prod.konflux-ocp-ci.dev" "10.133.0.31:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.58:59090 10.132.0.58:443 10.132.0.13:39014 - v1-chat-completions-path 2026-11-04T05:14:26.823252Z info ads ADS: new connection for node:3 2026-11-04T05:14:26.834355Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-04T05:14:26.834414Z info cache read certificate from file resource=file-root:system 2026-11-04T05:14:28.378274Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T05:14:48.319630Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T05:14:55.082885Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T05:14:55.082920Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T05:14:55.082931Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T05:14:55.082937Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T05:14:55.083138Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-04T05:14:54.920Z] "POST /anything/e2e-dft-485f75-fa0401 HTTP/1.1" 200 - via_upstream - "-" 109 1588 163 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "f0e086be-af27-4c18-8119-6efac727287e" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.58:51468 10.132.0.58:443 10.132.0.13:40706 - llm.external-model-e2e-dft-485f75-6c6d1a5cd295.0 2026-11-04T05:15:23.058003Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T05:16:04.906712Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T05:16:24.477512Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T05:16:29.697701Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T05:17:51.875838Z info ads ADS: "" 3 terminated with stream closed 2026-11-04T05:18:07.404009Z info ads ADS: new connection for node:4 2026-11-04T05:18:07.410235Z info cache read certificate from file resource=file-root:system 2026-11-04T05:18:08.554465Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T05:18:15.489793Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T05:18:21.487960Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T05:18:21.487998Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T05:18:21.488011Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T05:18:21.488016Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-04T05:18:21.335Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 66 308 152 - "44.212.242.249,10.132.0.13" "python-requests/2.34.2" "92b88ee1-29d0-4ddc-8e27-66954f489266" "maas.apps.0a41f24c8cfe96cff5be.prod.konflux-ocp-ci.dev" "10.134.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.58:60360 10.132.0.58:443 10.132.0.13:42358 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T05:18:33.795396Z info ads ADS: "" 4 terminated 2026-11-04T05:18:33.863737Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T05:18:37.079058Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8