2026-11-04T03:26:52.816028Z info FLAG: --concurrency="0" 2026-11-04T03:26:52.816049Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-04T03:26:52.816053Z info FLAG: --help="false" 2026-11-04T03:26:52.816055Z info FLAG: --log_as_json="false" 2026-11-04T03:26:52.816058Z info FLAG: --log_caller="" 2026-11-04T03:26:52.816060Z info FLAG: --log_output_level="default:info" 2026-11-04T03:26:52.816062Z info FLAG: --log_stacktrace_level="default:none" 2026-11-04T03:26:52.816066Z info FLAG: --log_target="[stdout]" 2026-11-04T03:26:52.816069Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-04T03:26:52.816071Z info FLAG: --outlierLogPath="" 2026-11-04T03:26:52.816073Z info FLAG: --profiling="true" 2026-11-04T03:26:52.816075Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-04T03:26:52.816077Z info FLAG: --proxyLogLevel="warning" 2026-11-04T03:26:52.816079Z info FLAG: --serviceCluster="istio-proxy" 2026-11-04T03:26:52.816081Z info FLAG: --stsPort="0" 2026-11-04T03:26:52.816083Z info FLAG: --templateFile="" 2026-11-04T03:26:52.816085Z info FLAG: --tokenManagerPlugin="" 2026-11-04T03:26:52.816088Z info FLAG: --vklog="0" 2026-11-04T03:26:52.816090Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-04T03:26:52.816240Z info Proxy role ips=[10.132.0.39] type=router id=maas-default-gateway-openshift-default-6dcf757899-5s8q9.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-04T03:26:52.816281Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-04T03:26:52.817446Z info cpu limit detected as 2, setting concurrency 2026-11-04T03:26:52.817708Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-04T03:26:52.817722Z info JWT policy is third-party-jwt 2026-11-04T03:26:52.817725Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-04T03:26:52.819463Z info Starting default Istio SDS Server 2026-11-04T03:26:52.819515Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-04T03:26:52.819540Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-04T03:26:52.819963Z info Opening status port 15020 2026-11-04T03:26:52.820916Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-04T03:26:52.821576Z info sds Starting SDS grpc server 2026-11-04T03:26:52.821620Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-04T03:26:52.822185Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-04T03:26:52.822996Z info Starting proxy agent 2026-11-04T03:26:52.823027Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-04T03:26:52.884515Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-04T03:26:52.909153Z info cache generated new workload certificate resourceName=default latency=87.450193ms ttl=23h59m59.090850486s 2026-11-04T03:26:52.909205Z info cache Root cert has changed, start rotating root cert 2026-11-04T03:26:52.909238Z info cache returned workload trust anchor from cache ttl=23h59m59.090763398s 2026-11-04T03:26:53.039869Z info ads ADS: new connection for node:1 2026-11-04T03:26:53.039998Z info cache returned workload certificate from cache ttl=23h59m58.960003906s 2026-11-04T03:26:53.040556Z info ads ADS: new connection for node:2 2026-11-04T03:26:53.040637Z info cache returned workload trust anchor from cache ttl=23h59m58.959364649s 2026-11-04T03:26:53.122511Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T03:27:01.213023Z info Readiness succeeded in 8.399785823s 2026-11-04T03:27:01.213392Z info Envoy proxy is ready 2026-11-04T03:27:50.754721Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:27:50.754762Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:27:50.754778Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:27:50.754784Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T03:27:50.579Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 320 175 - "44.212.242.249,10.132.0.18" "curl/7.76.1" "e89b00f4-09b4-434b-a3e0-d98ffda57756" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "10.133.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.39:60020 10.132.0.39:443 10.132.0.18:57632 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T03:27:53.816Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 55 - "44.212.242.249,10.132.0.18" "curl/7.76.1" "24e51b3f-8378-4548-8f09-daf910857ed2" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.39:443 10.132.0.18:57634 - v1-model-routing 2026-11-04T03:28:05.338517Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:28:05.338548Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:28:05.338560Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:28:05.338564Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T03:28:05.323Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 91 15 - "44.212.242.249,10.132.0.18" "curl/7.76.1" "dd9ad10f-e1e5-4b7c-a3ab-c137c9c43a13" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "10.133.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.39:60020 10.132.0.39:443 10.132.0.18:46672 - odh-ai-gateway-infra.maas-api-route.4 2026-11-04T03:32:42.860394Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T03:32:42.860438Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T03:32:42.860455Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T03:32:42.860462Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-04T03:32:42.734Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 84 326 125 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "5f6d6acf-c08f-4ad7-bd78-4fe36de0ae64" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "10.133.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.39:47162 10.132.0.39:443 10.132.0.18:52658 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T03:32:43.062Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 83 0 47 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "cef9e2d0-92d4-4458-a02b-b0b5696eaf0e" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.132.0.39:443 10.132.0.18:52670 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T03:32:43.422149Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:32:43.422184Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:32:43.422195Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:32:43.422200Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T03:32:43.891792Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:32:43.891824Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:32:43.891836Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:32:43.891841Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T03:32:43.310Z] "DELETE /v1/api-keys/69efc6cc-4a71-42f9-a1ef-efd3e9791e87 HTTP/1.1" 200 - via_upstream - "-" 0 370 111 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "27ac06f9-6abf-4813-923f-c3ddd4cc3c3e" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "10.133.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.39:47172 10.132.0.39:443 10.132.0.18:52682 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T03:32:43.785Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 106 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "ca5b5955-d6bc-4bce-b3ae-441b1cb4c0eb" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "10.133.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.39:47172 10.132.0.39:443 10.132.0.18:52696 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T03:32:46.509789Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:32:46.509822Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:32:46.509829Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-11-04T03:32:46.509840Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:32:46.509845Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T03:32:46.400Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 109 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "88c7f8bf-345f-4a10-a8fd-d6ef6121e1ee" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "10.133.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.39:47172 10.132.0.39:443 10.132.0.18:52708 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T03:32:47.875Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 85 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "2955a58b-774b-46bc-aff0-f2fcbfa0b26f" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.132.0.39:443 10.132.0.18:48230 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T03:32:49.344340Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T03:32:49.344373Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T03:32:49.344385Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T03:32:49.344389Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-04T03:32:49.236Z] "DELETE /v1/api-keys/3b31ab5a-bd40-44b7-9e94-2761959715b9 HTTP/1.1" 200 - via_upstream - "-" 0 321 108 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "1dfcf66d-5349-45d2-8a0c-44d580d6a822" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "10.133.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.39:47162 10.132.0.39:443 10.132.0.18:48250 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T03:32:52.616110Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-04T03:32:52.616139Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-04T03:32:52.616144Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-04T03:32:52.616148Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-11-04T03:32:52.580Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 77 35 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "1f623d48-c1f3-4215-b480-2b8dc782228a" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.39:443 10.132.0.18:48254 - v1-completions-path [2026-11-04T03:32:58.125Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 43 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "3b0c8bdb-6f0a-479f-9b03-5a203d8dcb53" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.39:443 10.132.0.18:42354 - v1-completions-path 2026-11-04T03:32:58.648447Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:32:58.648513Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:32:58.648526Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:32:58.648531Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T03:32:58.542Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 105 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "f26876c2-7eaa-4bfc-b41c-e3254ba6092a" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "10.133.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.39:47172 10.132.0.39:443 10.132.0.18:42360 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T03:32:58.959Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 284 75 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "2e2e9a17-ae13-4824-8e73-4ae51e535802" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.39:46708 10.132.0.39:443 10.132.0.18:42374 - v1-completions-path [2026-11-04T03:32:59.396Z] "POST /maas-api/internal/v1/subscriptions/select HTTP/1.1" 404 NR route_not_found - "-" 56 0 7 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "9fdb56a4-1688-4e02-83eb-b8cbd7a33803" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "-" - - 10.132.0.39:443 10.132.0.18:42384 - - 2026-11-04T03:33:00.854769Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:33:00.854809Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:33:00.854824Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:33:00.854829Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T03:33:00.749Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 105 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "3c629b17-a8b7-436f-9db5-084b126e1714" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "10.133.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.39:47172 10.132.0.39:443 10.132.0.18:42394 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T03:33:01.127Z] "POST /maas-api/v1/tokens HTTP/1.1" 401 - - "-" 20 88 20 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "9f32270c-4e71-4676-bb24-4e314310575b" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.132.0.39:443 10.132.0.18:42400 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T03:33:01.447381Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:33:01.447419Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:33:01.447435Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:33:01.447440Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T03:33:01.752158Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:33:01.752192Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:33:01.752204Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T03:33:01.752208Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:33:02.000980Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-11-04T03:33:01.344Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 102 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "bfc0647e-9158-4e9a-92fc-6ac9e3e6e116" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "10.133.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.39:47172 10.132.0.39:443 10.132.0.18:42406 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T03:33:01.610Z] "POST /maas-api/internal/v1/api-keys/cleanup HTTP/1.1" 404 NR route_not_found - "-" 2 0 11 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "f9cccf07-4633-41e1-95f0-43ec0bf77c92" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "-" - - 10.132.0.39:443 10.132.0.18:42422 - - [2026-11-04T03:33:01.647Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 86 328 104 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "2bca7b3c-b91a-4f66-9081-c833bf9d7cd7" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "10.133.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.39:47172 10.132.0.39:443 10.132.0.18:42424 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T03:33:01.873Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 135 2830 60 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "1ded5bfe-059e-44bc-9951-1f06b3146261" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.39:46718 10.132.0.39:443 10.132.0.18:42434 - v1-chat-completions-path [2026-11-04T03:33:01.942Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 136 364 58 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "a5f989b8-bb1c-442e-a364-b85c83f7918b" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.39:46718 10.132.0.39:443 10.132.0.18:42440 - v1-chat-completions-path 2026-11-04T03:33:02.587601Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T03:33:02.587633Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T03:33:02.587645Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T03:33:02.587650Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-04T03:33:02.479Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 107 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "6bd03ca6-f29b-4afe-9c61-dfd4e7347555" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "10.133.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.39:47162 10.132.0.39:443 10.132.0.18:42442 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T03:35:41.696718Z info ads ADS: new connection for node:3 2026-11-04T03:35:41.704214Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-04T03:35:41.704379Z info cache read certificate from file resource=file-root:system 2026-11-04T03:35:44.798747Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T03:35:58.348030Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T03:36:20.101157Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T03:36:30.127060Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:36:30.127096Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:36:30.127108Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T03:36:30.127113Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T03:36:29.954Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 66 308 173 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "3a07dea4-ef1e-4a6f-acbc-a8106f9f3389" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "10.133.0.43:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.39:55274 10.132.0.39:443 10.132.0.18:43986 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T03:36:57.663912Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T03:37:55.520158Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T03:39:09.408Z] "POST /llm/e2e-dft-b3172b/anything/e2e-dft-b3172b-51f489 HTTP/1.1" 503 - - "-" 109 0 72 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "4f7705c6-5676-43b8-939c-6ff2c31e6e12" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-b3172b.llm.svc.cluster.local - 10.132.0.39:443 10.132.0.18:47710 - llm.external-model-e2e-dft-b3172b-b5f5747897a5.2 [2026-11-04T03:39:41.237Z] "POST /llm/e2e-dft-b3172b/anything/e2e-dft-b3172b-51f489 HTTP/1.1" 503 - - "-" 109 0 64 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "c0b4a7ea-dec0-42d4-8d75-a256a93c839b" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-b3172b.llm.svc.cluster.local - 10.132.0.39:443 10.132.0.18:36212 - llm.external-model-e2e-dft-b3172b-b5f5747897a5.2 [2026-11-04T03:40:02.507Z] "POST /llm/e2e-dft-b3172b/anything/e2e-dft-b3172b-51f489 HTTP/1.1" 503 - - "-" 109 0 70 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "2bb6f05f-d090-4714-a0f8-cd3467b460bc" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-b3172b.llm.svc.cluster.local - 10.132.0.39:443 10.132.0.18:33278 - llm.external-model-e2e-dft-b3172b-b5f5747897a5.2 [2026-11-04T03:40:13.082Z] "POST /llm/e2e-dft-b3172b/anything/e2e-dft-b3172b-51f489 HTTP/1.1" 503 - - "-" 109 0 66 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "54f107fc-a18c-4def-9cf9-a7cba8ec5206" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-b3172b.llm.svc.cluster.local - 10.132.0.39:443 10.132.0.18:55048 - llm.external-model-e2e-dft-b3172b-b5f5747897a5.2 2026-11-04T03:40:35.055350Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T03:40:48.772756Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T03:40:48.772787Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T03:40:48.772798Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T03:40:48.772804Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T03:40:48.773011Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-04T03:40:48.645Z] "POST /anything/e2e-dft-b3172b-51f489 HTTP/1.1" 200 - via_upstream - "-" 109 1593 127 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "15d778fd-32f6-445e-86b1-490d06138650" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.39:42912 10.132.0.39:443 10.132.0.18:33352 - llm.external-model-e2e-dft-b3172b-b5f5747897a5.0 2026-11-04T03:41:07.592627Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-04T03:41:37.705Z] "POST /llm/e2e-dft-d9a10e/anything/e2e-dft-d9a10e-342eba HTTP/1.1" 503 - - "-" 109 0 73 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "e6c6de3b-dbe9-4791-85de-cf01d61de17a" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-d9a10e.llm.svc.cluster.local - 10.132.0.39:443 10.132.0.18:52280 - llm.external-model-e2e-dft-d9a10e-9716c7f69aa6.2 2026-11-04T03:42:20.179158Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-04T03:42:20.115Z] "POST /llm/e2e-dft-d9a10e/anything/e2e-dft-d9a10e-342eba HTTP/1.1" 503 - - "-" 109 0 66 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "4c6d5738-cbd3-4537-afaa-cbedbadfdc92" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-d9a10e.llm.svc.cluster.local - 10.132.0.39:443 10.132.0.18:49608 - llm.external-model-e2e-dft-d9a10e-9716c7f69aa6.2 2026-11-04T03:42:25.441814Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 [2026-11-04T03:42:25.379Z] "POST /llm/e2e-dft-d9a10e/anything/e2e-dft-d9a10e-342eba HTTP/1.1" 503 - - "-" 109 0 62 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "3e5d47a8-2e74-4c82-b7e0-69717063a529" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-d9a10e.llm.svc.cluster.local - 10.132.0.39:443 10.132.0.18:49626 - llm.external-model-e2e-dft-d9a10e-9716c7f69aa6.2 [2026-11-04T03:42:36.088Z] "POST /llm/e2e-dft-d9a10e/anything/e2e-dft-d9a10e-342eba HTTP/1.1" 503 - - "-" 109 0 62 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "30fefbb4-3be5-428e-8018-bbf3eaab264b" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-d9a10e.llm.svc.cluster.local - 10.132.0.39:443 10.132.0.18:51252 - llm.external-model-e2e-dft-d9a10e-9716c7f69aa6.2 [2026-11-04T03:42:41.350Z] "POST /llm/e2e-dft-d9a10e/anything/e2e-dft-d9a10e-342eba HTTP/1.1" 503 - - "-" 109 0 69 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "465a5846-874d-4296-8754-be4986dff507" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-d9a10e.llm.svc.cluster.local - 10.132.0.39:443 10.132.0.18:48002 - llm.external-model-e2e-dft-d9a10e-9716c7f69aa6.2 [2026-11-04T03:43:02.447Z] "POST /llm/e2e-dft-d9a10e/anything/e2e-dft-d9a10e-342eba HTTP/1.1" 503 - - "-" 109 0 65 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "7226cf08-b3d4-42a2-89ea-88addb046b0f" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-d9a10e.llm.svc.cluster.local - 10.132.0.39:443 10.132.0.18:44350 - llm.external-model-e2e-dft-d9a10e-9716c7f69aa6.2 [2026-11-04T03:43:55.527Z] "POST /llm/e2e-dft-d9a10e/anything/e2e-dft-d9a10e-342eba HTTP/1.1" 503 - - "-" 109 0 63 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "7e5fff5c-a668-4b5c-8581-ac8d763274f9" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-d9a10e.llm.svc.cluster.local - 10.132.0.39:443 10.132.0.18:46490 - llm.external-model-e2e-dft-d9a10e-9716c7f69aa6.2 [2026-11-04T03:45:09.627Z] "POST /llm/e2e-dft-d9a10e/anything/e2e-dft-d9a10e-342eba HTTP/1.1" 503 - - "-" 109 0 64 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "c426c14e-c50b-43d8-9f00-1d5ae72b3a1e" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-d9a10e.llm.svc.cluster.local - 10.132.0.39:443 10.132.0.18:39924 - llm.external-model-e2e-dft-d9a10e-9716c7f69aa6.2 [2026-11-04T03:45:41.464Z] "POST /llm/e2e-dft-d9a10e/anything/e2e-dft-d9a10e-342eba HTTP/1.1" 503 - - "-" 109 0 63 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "b512de61-a3b0-43e2-ac8f-77ef9224da7e" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-d9a10e.llm.svc.cluster.local - 10.132.0.39:443 10.132.0.18:32820 - llm.external-model-e2e-dft-d9a10e-9716c7f69aa6.2 [2026-11-04T03:45:52.179Z] "POST /llm/e2e-dft-d9a10e/anything/e2e-dft-d9a10e-342eba HTTP/1.1" 503 - - "-" 109 0 69 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "8c52f5ec-9aa3-4d72-a8c3-e5001dadfa0f" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-d9a10e.llm.svc.cluster.local - 10.132.0.39:443 10.132.0.18:38394 - llm.external-model-e2e-dft-d9a10e-9716c7f69aa6.2 [2026-11-04T03:45:57.449Z] "POST /llm/e2e-dft-d9a10e/anything/e2e-dft-d9a10e-342eba HTTP/1.1" 503 - - "-" 109 0 62 - "44.212.242.249,10.132.0.18" "python-requests/2.34.2" "60f2808a-ca1e-4fce-b019-0da6324c9e60" "maas.apps.7c7307434aa4d41108be.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-d9a10e.llm.svc.cluster.local - 10.132.0.39:443 10.132.0.18:55470 - llm.external-model-e2e-dft-d9a10e-9716c7f69aa6.2 2026-11-04T03:46:21.514972Z info ads ADS: "" 3 terminated 2026-11-04T03:46:22.733688Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-04T03:46:27.058800Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8