2026-11-04T01:35:43.746746Z info FLAG: --concurrency="0" 2026-11-04T01:35:43.746765Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-04T01:35:43.746769Z info FLAG: --help="false" 2026-11-04T01:35:43.746772Z info FLAG: --log_as_json="false" 2026-11-04T01:35:43.746774Z info FLAG: --log_caller="" 2026-11-04T01:35:43.746776Z info FLAG: --log_output_level="default:info" 2026-11-04T01:35:43.746778Z info FLAG: --log_stacktrace_level="default:none" 2026-11-04T01:35:43.746782Z info FLAG: --log_target="[stdout]" 2026-11-04T01:35:43.746785Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-04T01:35:43.746787Z info FLAG: --outlierLogPath="" 2026-11-04T01:35:43.746789Z info FLAG: --profiling="true" 2026-11-04T01:35:43.746791Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-04T01:35:43.746793Z info FLAG: --proxyLogLevel="warning" 2026-11-04T01:35:43.746796Z info FLAG: --serviceCluster="istio-proxy" 2026-11-04T01:35:43.746798Z info FLAG: --stsPort="0" 2026-11-04T01:35:43.746800Z info FLAG: --templateFile="" 2026-11-04T01:35:43.746802Z info FLAG: --tokenManagerPlugin="" 2026-11-04T01:35:43.746804Z info FLAG: --vklog="0" 2026-11-04T01:35:43.746807Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-04T01:35:43.746962Z info Proxy role ips=[10.133.0.67] type=router id=maas-default-gateway-openshift-default-5db77fbd9f-fn6tt.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-04T01:35:43.747002Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-04T01:35:43.748018Z info cpu limit detected as 2, setting concurrency 2026-11-04T01:35:43.748624Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-04T01:35:43.748639Z info JWT policy is third-party-jwt 2026-11-04T01:35:43.748643Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-04T01:35:43.749882Z info Starting default Istio SDS Server 2026-11-04T01:35:43.749904Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-04T01:35:43.749923Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-04T01:35:43.750293Z info Opening status port 15020 2026-11-04T01:35:43.751653Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-04T01:35:43.752242Z info sds Starting SDS grpc server 2026-11-04T01:35:43.752261Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-04T01:35:43.753080Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-04T01:35:43.753995Z info Starting proxy agent 2026-11-04T01:35:43.754079Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-04T01:35:43.822167Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-04T01:35:43.857824Z info cache generated new workload certificate resourceName=default latency=105.409672ms ttl=23h59m59.142181587s 2026-11-04T01:35:43.857869Z info cache Root cert has changed, start rotating root cert 2026-11-04T01:35:43.857911Z info cache returned workload trust anchor from cache ttl=23h59m59.142090042s 2026-11-04T01:35:44.076290Z info ads ADS: new connection for node:1 2026-11-04T01:35:44.082773Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-04T01:35:44.082818Z info cache read certificate from file resource=file-root:system 2026-11-04T01:35:44.114100Z info ads ADS: new connection for node:2 2026-11-04T01:35:44.114166Z info cache returned workload certificate from cache ttl=23h59m58.885835703s 2026-11-04T01:35:44.115193Z info ads ADS: new connection for node:3 2026-11-04T01:35:44.115256Z info cache returned workload trust anchor from cache ttl=23h59m58.884744926s 2026-11-04T01:35:44.249076Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:35:52.584340Z info Readiness succeeded in 8.840315485s 2026-11-04T01:35:52.584788Z info Envoy proxy is ready 2026-11-04T01:35:56.735880Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:37:46.619Z] "POST /llm/e2e-dft-c70904/anything/e2e-dft-c70904-9212d8 HTTP/1.1" 503 - - "-" 109 0 83 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "8b148b92-05ef-4623-86cc-5556a0cffa07" "maas.apps.0408108e6a0fa65d217a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-c70904.llm.svc.cluster.local - 10.133.0.67:443 10.132.0.10:49686 - llm.external-model-e2e-dft-c70904-eb4d98a8f8c8.2 [2026-11-04T01:38:39.668Z] "POST /llm/e2e-dft-c70904/anything/e2e-dft-c70904-9212d8 HTTP/1.1" 503 - - "-" 109 0 77 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "a0c44d07-c6d1-4210-8ae7-26e354fef4c5" "maas.apps.0408108e6a0fa65d217a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-c70904.llm.svc.cluster.local - 10.133.0.67:443 10.132.0.10:52456 - llm.external-model-e2e-dft-c70904-eb4d98a8f8c8.2 2026-11-04T01:39:32.148063Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T01:39:32.870Z] "POST /llm/e2e-dft-c70904/anything/e2e-dft-c70904-9212d8 HTTP/1.1" 503 - - "-" 109 0 99 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "fab952b6-e23a-429b-bd3f-f7393447ddcf" "maas.apps.0408108e6a0fa65d217a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-c70904.llm.svc.cluster.local - 10.133.0.67:443 10.132.0.10:57396 - llm.external-model-e2e-dft-c70904-eb4d98a8f8c8.2 2026-11-04T01:40:12.573553Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:40:53.034631Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:41:19.194828Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:41:19.194869Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:41:19.194885Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:41:19.194893Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T01:41:19.016Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 66 308 178 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "72f9b54c-78cc-4e3b-8a61-47066dc6471f" "maas.apps.0408108e6a0fa65d217a.prod.konflux-ocp-ci.dev" "10.134.0.27:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.67:35638 10.133.0.67:443 10.132.0.10:48680 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T01:41:46.317Z] "POST /llm/e2e-dft-b7777b/anything/e2e-dft-b7777b-0254f7 HTTP/1.1" 503 - - "-" 109 0 76 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "b560147a-f7cf-4dfe-9001-4aa76727513b" "maas.apps.0408108e6a0fa65d217a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-b7777b.llm.svc.cluster.local - 10.133.0.67:443 10.132.0.10:57858 - llm.external-model-e2e-dft-b7777b-3065d3c15d7b.2 [2026-11-04T01:42:07.779Z] "POST /llm/e2e-dft-b7777b/anything/e2e-dft-b7777b-0254f7 HTTP/1.1" 503 - - "-" 109 0 68 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "0ba71e16-4fdf-400b-b305-99bb34b94b9a" "maas.apps.0408108e6a0fa65d217a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-b7777b.llm.svc.cluster.local - 10.133.0.67:443 10.132.0.10:38434 - llm.external-model-e2e-dft-b7777b-3065d3c15d7b.2 [2026-11-04T01:42:28.950Z] "POST /llm/e2e-dft-b7777b/anything/e2e-dft-b7777b-0254f7 HTTP/1.1" 503 - - "-" 109 0 69 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "b6370157-3284-49ff-9401-bf5a1d50cf5b" "maas.apps.0408108e6a0fa65d217a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-b7777b.llm.svc.cluster.local - 10.133.0.67:443 10.132.0.10:57838 - llm.external-model-e2e-dft-b7777b-3065d3c15d7b.2 2026-11-04T01:42:34.290080Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-11-04T01:42:34.231Z] "POST /llm/e2e-dft-b7777b/anything/e2e-dft-b7777b-0254f7 HTTP/1.1" 503 - - "-" 109 0 61 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "d7ebc756-bc7e-478a-932d-45e4c4ae9271" "maas.apps.0408108e6a0fa65d217a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-b7777b.llm.svc.cluster.local - 10.133.0.67:443 10.132.0.10:57852 - llm.external-model-e2e-dft-b7777b-3065d3c15d7b.2 [2026-11-04T01:44:36.300Z] "POST /llm/e2e-dft-b7777b/anything/e2e-dft-b7777b-0254f7 HTTP/1.1" 503 - - "-" 109 0 70 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "055f73d8-eae1-4b56-8d20-1bb86a0ebdfa" "maas.apps.0408108e6a0fa65d217a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-b7777b.llm.svc.cluster.local - 10.133.0.67:443 10.132.0.10:36704 - llm.external-model-e2e-dft-b7777b-3065d3c15d7b.2 [2026-11-04T01:45:34.714Z] "POST /llm/e2e-dft-b7777b/anything/e2e-dft-b7777b-0254f7 HTTP/1.1" 503 - - "-" 109 0 67 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "23494a07-f30f-420c-bb04-7c38efdde203" "maas.apps.0408108e6a0fa65d217a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-b7777b.llm.svc.cluster.local - 10.133.0.67:443 10.132.0.10:37930 - llm.external-model-e2e-dft-b7777b-3065d3c15d7b.2 [2026-11-04T01:46:11.783Z] "POST /llm/e2e-dft-b7777b/anything/e2e-dft-b7777b-0254f7 HTTP/1.1" 503 - - "-" 109 0 68 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "d194017f-30a5-4b03-a4ed-6bbfe8f8898a" "maas.apps.0408108e6a0fa65d217a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-b7777b.llm.svc.cluster.local - 10.133.0.67:443 10.132.0.10:47412 - llm.external-model-e2e-dft-b7777b-3065d3c15d7b.2 2026-11-04T01:46:38.478849Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-04T01:46:38.397Z] "POST /llm/e2e-dft-b7777b/anything/e2e-dft-b7777b-0254f7 HTTP/1.1" 503 - - "-" 109 0 84 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "6a2f636a-e766-47cf-8975-df8629fa75e8" "maas.apps.0408108e6a0fa65d217a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-b7777b.llm.svc.cluster.local - 10.133.0.67:443 10.132.0.10:40764 - llm.external-model-e2e-dft-b7777b-3065d3c15d7b.2 [2026-11-04T01:46:43.709Z] "POST /llm/e2e-dft-b7777b/anything/e2e-dft-b7777b-0254f7 HTTP/1.1" 503 - - "-" 109 0 60 - "34.228.250.194,10.132.0.10" "python-requests/2.34.2" "a1c608e0-ed7f-4584-8fd0-a2c7f0a00126" "maas.apps.0408108e6a0fa65d217a.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-b7777b.llm.svc.cluster.local - 10.133.0.67:443 10.132.0.10:40774 - llm.external-model-e2e-dft-b7777b-3065d3c15d7b.2 2026-11-04T01:46:57.304079Z info ads ADS: "" 1 terminated 2026-11-04T01:46:58.498091Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T01:47:02.933295Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8