2026-11-04T01:54:02.048543Z info FLAG: --concurrency="0" 2026-11-04T01:54:02.048562Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-04T01:54:02.048566Z info FLAG: --help="false" 2026-11-04T01:54:02.048568Z info FLAG: --log_as_json="false" 2026-11-04T01:54:02.048570Z info FLAG: --log_caller="" 2026-11-04T01:54:02.048572Z info FLAG: --log_output_level="default:info" 2026-11-04T01:54:02.048574Z info FLAG: --log_stacktrace_level="default:none" 2026-11-04T01:54:02.048579Z info FLAG: --log_target="[stdout]" 2026-11-04T01:54:02.048581Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-04T01:54:02.048583Z info FLAG: --outlierLogPath="" 2026-11-04T01:54:02.048585Z info FLAG: --profiling="true" 2026-11-04T01:54:02.048587Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-04T01:54:02.048589Z info FLAG: --proxyLogLevel="warning" 2026-11-04T01:54:02.048591Z info FLAG: --serviceCluster="istio-proxy" 2026-11-04T01:54:02.048593Z info FLAG: --stsPort="0" 2026-11-04T01:54:02.048595Z info FLAG: --templateFile="" 2026-11-04T01:54:02.048597Z info FLAG: --tokenManagerPlugin="" 2026-11-04T01:54:02.048600Z info FLAG: --vklog="0" 2026-11-04T01:54:02.048602Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-04T01:54:02.048806Z info Proxy role ips=[10.132.0.45] type=router id=maas-default-gateway-openshift-default-7f694df8bc-nmcxv.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-04T01:54:02.048860Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-04T01:54:02.050068Z info cpu limit detected as 2, setting concurrency 2026-11-04T01:54:02.050326Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-04T01:54:02.050337Z info JWT policy is third-party-jwt 2026-11-04T01:54:02.050340Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-04T01:54:02.052863Z info Starting default Istio SDS Server 2026-11-04T01:54:02.052889Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-04T01:54:02.052919Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-04T01:54:02.053174Z info Opening status port 15020 2026-11-04T01:54:02.053975Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-04T01:54:02.055560Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-04T01:54:02.056423Z info Starting proxy agent 2026-11-04T01:54:02.056459Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-04T01:54:02.065283Z info sds Starting SDS grpc server 2026-11-04T01:54:02.065306Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-04T01:54:02.120673Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-04T01:54:02.211088Z info cache generated new workload certificate resourceName=default latency=156.84977ms ttl=23h59m59.78891588s 2026-11-04T01:54:02.211134Z info cache Root cert has changed, start rotating root cert 2026-11-04T01:54:02.211174Z info cache returned workload trust anchor from cache ttl=23h59m59.788826463s 2026-11-04T01:54:02.254533Z info ads ADS: new connection for node:1 2026-11-04T01:54:02.254607Z info cache returned workload certificate from cache ttl=23h59m59.745395349s 2026-11-04T01:54:02.255554Z info ads ADS: new connection for node:2 2026-11-04T01:54:02.255644Z info cache returned workload trust anchor from cache ttl=23h59m59.744357175s 2026-11-04T01:54:02.344397Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:54:10.472879Z info Readiness succeeded in 8.426734422s 2026-11-04T01:54:10.473366Z info Envoy proxy is ready 2026-11-04T01:57:00.265509Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:57:00.265545Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:57:00.265557Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:57:00.265562Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T01:57:00.169Z] "DELETE /v1/api-keys/2109b11a-a334-4e8c-8faf-85a78c8f6562 HTTP/1.1" 200 - via_upstream - "-" 0 370 96 - "34.228.250.194,10.132.0.7" "python-requests/2.34.2" "d3a5d21b-40b7-4bad-843b-bbb592e81e6e" "maas.apps.4a263ef06a303c77cb9a.prod.konflux-ocp-ci.dev" "10.133.0.35:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.45:59702 10.132.0.45:443 10.132.0.7:50734 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T01:57:00.546326Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:57:00.546356Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:57:00.546367Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:57:00.546372Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T01:57:00.473Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 72 - "34.228.250.194,10.132.0.7" "python-requests/2.34.2" "4e99b81c-bb1a-4bae-b3b2-98ec1b79f702" "maas.apps.4a263ef06a303c77cb9a.prod.konflux-ocp-ci.dev" "10.133.0.35:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.45:59702 10.132.0.45:443 10.132.0.7:50738 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T01:57:01.736314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:57:01.736349Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:57:01.736361Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T01:57:01.736366Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:57:02.177356Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:57:02.177390Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:57:02.177403Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T01:57:02.177407Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:57:02.454289Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:57:02.454322Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:57:02.454333Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-04T01:57:02.454337Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-04T01:57:01.658Z] "DELETE /v1/api-keys/52a68570-d56c-4145-adc0-5b56f673a168 HTTP/1.1" 200 - via_upstream - "-" 0 321 77 - "34.228.250.194,10.132.0.7" "python-requests/2.34.2" "303c6139-79de-4d59-89c4-ee96d4b32990" "maas.apps.4a263ef06a303c77cb9a.prod.konflux-ocp-ci.dev" "10.133.0.35:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.45:59704 10.132.0.45:443 10.132.0.7:50756 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T01:57:02.103Z] "DELETE /v1/api-keys/18df4068-7a81-4f4a-99e2-a9a4abd76ac9 HTTP/1.1" 200 - via_upstream - "-" 0 321 73 - "34.228.250.194,10.132.0.7" "python-requests/2.34.2" "c31f3f46-f2f6-4570-9468-d81fcbbeb95f" "maas.apps.4a263ef06a303c77cb9a.prod.konflux-ocp-ci.dev" "10.133.0.35:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.45:59702 10.132.0.45:443 10.132.0.7:50768 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T01:57:02.204Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 57 - "34.228.250.194,10.132.0.7" "python-requests/2.34.2" "ce1e991a-07aa-4723-973e-63a4aa403530" "maas.apps.4a263ef06a303c77cb9a.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.132.0.45:443 10.132.0.7:50776 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T01:57:02.381Z] "DELETE /v1/api-keys/09940d01-feb4-44e7-9766-3baac7f27ee1 HTTP/1.1" 200 - via_upstream - "-" 0 321 72 - "34.228.250.194,10.132.0.7" "python-requests/2.34.2" "333d37b9-d10e-4576-9a05-70c26e7535fd" "maas.apps.4a263ef06a303c77cb9a.prod.konflux-ocp-ci.dev" "10.133.0.35:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.45:59702 10.132.0.45:443 10.132.0.7:50788 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T01:57:02.718513Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:57:02.718544Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:57:02.718550Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 2026-11-04T01:57:02.718561Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-04T01:57:02.718565Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-04T01:57:02.649Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 69 - "34.228.250.194,10.132.0.7" "python-requests/2.34.2" "6aa99397-41fe-4761-a17c-1efe9862cbdf" "maas.apps.4a263ef06a303c77cb9a.prod.konflux-ocp-ci.dev" "10.133.0.35:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.45:59702 10.132.0.45:443 10.132.0.7:50802 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T01:57:05.642705Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:57:05.642741Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:57:05.642755Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T01:57:05.642760Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-04T01:57:05.568Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 74 - "34.228.250.194,10.132.0.7" "python-requests/2.34.2" "13a56eab-77a4-4314-99d2-47c5cc37e9de" "maas.apps.4a263ef06a303c77cb9a.prod.konflux-ocp-ci.dev" "10.133.0.35:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.45:59704 10.132.0.45:443 10.132.0.7:56648 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-04T01:57:07.778Z] "POST /maas-api/internal/v1/subscriptions/select HTTP/1.1" 404 NR route_not_found - "-" 56 0 13 - "34.228.250.194,10.132.0.7" "python-requests/2.34.2" "c6be8ce6-c04d-4ab4-a651-9c43a6d56031" "maas.apps.4a263ef06a303c77cb9a.prod.konflux-ocp-ci.dev" "-" - - 10.132.0.45:443 10.132.0.7:56656 - - 2026-11-04T01:57:10.949883Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:57:10.949920Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-04T01:57:10.949935Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-04T01:57:10.949941Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-04T01:57:10.427Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 288 103 - "34.228.250.194,10.132.0.7" "python-requests/2.34.2" "0beef9bd-e052-47c1-9ee6-073b292fc346" "maas.apps.4a263ef06a303c77cb9a.prod.konflux-ocp-ci.dev" "10.133.0.39:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.45:42726 10.132.0.45:443 10.132.0.7:56666 - v1-completions-path [2026-11-04T01:57:10.851Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 98 - "34.228.250.194,10.132.0.7" "python-requests/2.34.2" "fd413f08-3892-431b-9baf-265eef999f67" "maas.apps.4a263ef06a303c77cb9a.prod.konflux-ocp-ci.dev" "10.133.0.35:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.45:59704 10.132.0.45:443 10.132.0.7:56674 - odh-ai-gateway-infra.maas-api-route.3 2026-11-04T01:58:18.036741Z info ads ADS: new connection for node:3 2026-11-04T01:58:18.047821Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-04T01:58:18.047941Z info cache read certificate from file resource=file-root:system 2026-11-04T01:58:22.024912Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:58:29.322984Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:58:42.906360Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T01:59:23.912654Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:00:03.433358Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:00:23.288948Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:00:28.016251Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:00:43.802810Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=9 2026-11-04T02:01:38.341291Z info ads ADS: "" 3 terminated with stream closed