2026-11-03T20:04:19.033014Z info FLAG: --concurrency="0" 2026-11-03T20:04:19.033033Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T20:04:19.033037Z info FLAG: --help="false" 2026-11-03T20:04:19.033039Z info FLAG: --log_as_json="false" 2026-11-03T20:04:19.033041Z info FLAG: --log_caller="" 2026-11-03T20:04:19.033044Z info FLAG: --log_output_level="default:info" 2026-11-03T20:04:19.033046Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T20:04:19.033050Z info FLAG: --log_target="[stdout]" 2026-11-03T20:04:19.033052Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T20:04:19.033054Z info FLAG: --outlierLogPath="" 2026-11-03T20:04:19.033058Z info FLAG: --profiling="true" 2026-11-03T20:04:19.033060Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T20:04:19.033062Z info FLAG: --proxyLogLevel="warning" 2026-11-03T20:04:19.033064Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T20:04:19.033066Z info FLAG: --stsPort="0" 2026-11-03T20:04:19.033068Z info FLAG: --templateFile="" 2026-11-03T20:04:19.033070Z info FLAG: --tokenManagerPlugin="" 2026-11-03T20:04:19.033072Z info FLAG: --vklog="0" 2026-11-03T20:04:19.033075Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T20:04:19.033246Z info Proxy role ips=[10.132.0.56] type=router id=maas-default-gateway-openshift-default-67bf986f99-wgkhr.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T20:04:19.033298Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T20:04:19.034319Z info cpu limit detected as 2, setting concurrency 2026-11-03T20:04:19.034928Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T20:04:19.034940Z info JWT policy is third-party-jwt 2026-11-03T20:04:19.034943Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T20:04:19.036174Z info Starting default Istio SDS Server 2026-11-03T20:04:19.036199Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T20:04:19.036230Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T20:04:19.036676Z info Opening status port 15020 2026-11-03T20:04:19.037975Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T20:04:19.039276Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T20:04:19.040068Z info Starting proxy agent 2026-11-03T20:04:19.040102Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T20:04:19.049331Z info sds Starting SDS grpc server 2026-11-03T20:04:19.049352Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T20:04:19.102041Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T20:04:19.123548Z info cache generated new workload certificate resourceName=default latency=85.311503ms ttl=23h59m59.876455879s 2026-11-03T20:04:19.123582Z info cache Root cert has changed, start rotating root cert 2026-11-03T20:04:19.123615Z info cache returned workload trust anchor from cache ttl=23h59m59.876385632s 2026-11-03T20:04:19.206412Z info ads ADS: new connection for node:1 2026-11-03T20:04:19.206491Z info cache returned workload certificate from cache ttl=23h59m59.793516482s 2026-11-03T20:04:19.206769Z info ads ADS: new connection for node:2 2026-11-03T20:04:19.206840Z info cache returned workload trust anchor from cache ttl=23h59m59.79316161s 2026-11-03T20:04:19.299822Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:04:28.437954Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:04:29.295339Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T20:04:29.716350Z warn Envoy proxy is NOT ready: Get "http://127.0.0.1:15000/stats?usedonly&filter=^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2026-11-03T20:04:31.674316Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:04:34.072857Z info Readiness succeeded in 15.042389796s 2026-11-03T20:04:34.073174Z info Readiness succeeded in 15.042713642s 2026-11-03T20:04:34.073407Z info Envoy proxy is ready 2026-11-03T20:04:44.060088Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:04:54.205529Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:05:04.441992Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:05:09.019333Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:05:12.385344Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:05:16.101614Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:06:57.078433Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T20:06:57.078469Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T20:06:57.078482Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T20:06:57.078486Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T20:06:57.008Z] "DELETE /v1/api-keys/36d6a2e2-e0fb-48af-9f76-38020f8bbea9 HTTP/1.1" 200 - via_upstream - "-" 0 335 70 - "34.228.250.194,10.132.0.15" "python-requests/2.34.2" "fcf2bced-8ccf-4665-8c1c-5d955bb02988" "maas.apps.93c989c08f3213195ddb.prod.konflux-ocp-ci.dev" "10.133.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.56:34668 10.132.0.56:443 10.132.0.15:51382 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T20:07:02.770880Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:07:09.683516Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:07:13.913068Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:07:29.897515Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:07:40.292188Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:07:50.467203Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:07:54.621265Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:08:00.640664Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:08:05.964836Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T20:08:19.529Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 35 0 33 - "34.228.250.194,10.132.0.15" "python-requests/2.34.2" "ffd8b444-18ed-43fb-8c3d-859cbeafd89f" "maas.apps.93c989c08f3213195ddb.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.132.0.56:443 10.132.0.15:54942 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T20:08:40.649713Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T20:08:40.649750Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T20:08:40.649761Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T20:08:40.649766Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T20:08:40.569Z] "POST /v1/api-keys/search HTTP/1.1" 200 - via_upstream - "-" 58 44 80 - "34.228.250.194,10.132.0.15" "python-requests/2.34.2" "6ceb882d-78e8-403e-a325-c7c66c1d3993" "maas.apps.93c989c08f3213195ddb.prod.konflux-ocp-ci.dev" "10.133.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.56:37232 10.132.0.56:443 10.132.0.15:45024 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T20:08:42.171335Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T20:08:42.171368Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T20:08:42.171380Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T20:08:42.171385Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T20:08:42.294090Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T20:08:42.294121Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T20:08:42.294165Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T20:08:42.294170Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T20:08:42.109Z] "GET /v1/api-keys/a759a46b-96be-4055-89f4-8775f762480a HTTP/1.1" 200 - via_upstream - "-" 0 291 61 - "34.228.250.194,10.132.0.15" "python-requests/2.34.2" "51f13614-1b5b-4aa4-92c4-ac36ebad0f09" "maas.apps.93c989c08f3213195ddb.prod.konflux-ocp-ci.dev" "10.133.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.56:37244 10.132.0.56:443 10.132.0.15:45030 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T20:08:42.233Z] "POST /v1/api-keys/bulk-revoke HTTP/1.1" 200 - via_upstream - "-" 57 125 60 - "34.228.250.194,10.132.0.15" "python-requests/2.34.2" "ebb4aafd-193a-4dfe-b6bb-97653a8945c9" "maas.apps.93c989c08f3213195ddb.prod.konflux-ocp-ci.dev" "10.133.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.56:37244 10.132.0.56:443 10.132.0.15:45046 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T20:08:58.295082Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:09:02.639603Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:09:17.989112Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:09:28.327760Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:09:38.490965Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T20:09:43.332954Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T20:09:55.709Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 42 0 42 - "34.228.250.194,10.132.0.15" "python-requests/2.34.2" "74d28acd-9c63-49d2-b52e-2c2767c13e58" "maas.apps.93c989c08f3213195ddb.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.132.0.56:443 10.132.0.15:49754 - odh-ai-gateway-infra.maas-api-route.3