2026-10-30T21:27:25.846938Z info FLAG: --concurrency="0" 2026-10-30T21:27:25.846959Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-30T21:27:25.846963Z info FLAG: --help="false" 2026-10-30T21:27:25.846966Z info FLAG: --log_as_json="false" 2026-10-30T21:27:25.846968Z info FLAG: --log_caller="" 2026-10-30T21:27:25.846970Z info FLAG: --log_output_level="default:info" 2026-10-30T21:27:25.846972Z info FLAG: --log_stacktrace_level="default:none" 2026-10-30T21:27:25.846976Z info FLAG: --log_target="[stdout]" 2026-10-30T21:27:25.846978Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-30T21:27:25.846980Z info FLAG: --outlierLogPath="" 2026-10-30T21:27:25.846982Z info FLAG: --profiling="true" 2026-10-30T21:27:25.846985Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-30T21:27:25.846987Z info FLAG: --proxyLogLevel="warning" 2026-10-30T21:27:25.846990Z info FLAG: --serviceCluster="istio-proxy" 2026-10-30T21:27:25.846992Z info FLAG: --stsPort="0" 2026-10-30T21:27:25.846994Z info FLAG: --templateFile="" 2026-10-30T21:27:25.846995Z info FLAG: --tokenManagerPlugin="" 2026-10-30T21:27:25.846998Z info FLAG: --vklog="0" 2026-10-30T21:27:25.847000Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-30T21:27:25.847149Z info Proxy role ips=[10.134.0.243] type=router id=maas-default-gateway-openshift-default-769df7846c-8nzgh.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-30T21:27:25.847195Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-30T21:27:25.848378Z info cpu limit detected as 2, setting concurrency 2026-10-30T21:27:25.848575Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-30T21:27:25.848586Z info JWT policy is third-party-jwt 2026-10-30T21:27:25.848589Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-30T21:27:25.849918Z info Starting default Istio SDS Server 2026-10-30T21:27:25.849942Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-30T21:27:25.849958Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-30T21:27:25.850354Z info Opening status port 15020 2026-10-30T21:27:25.851608Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-30T21:27:25.851956Z info sds Starting SDS grpc server 2026-10-30T21:27:25.851990Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-30T21:27:25.852850Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-30T21:27:25.853643Z info Starting proxy agent 2026-10-30T21:27:25.853678Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-30T21:27:25.916919Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-30T21:27:26.006890Z info cache generated new workload certificate resourceName=default latency=154.768211ms ttl=23h59m59.993117326s 2026-10-30T21:27:26.006941Z info cache Root cert has changed, start rotating root cert 2026-10-30T21:27:26.006985Z info cache returned workload trust anchor from cache ttl=23h59m59.993016035s 2026-10-30T21:27:26.067721Z info ads ADS: new connection for node:1 2026-10-30T21:27:26.067813Z info cache returned workload certificate from cache ttl=23h59m59.932189756s 2026-10-30T21:27:26.068083Z info ads ADS: new connection for node:2 2026-10-30T21:27:26.068171Z info cache returned workload trust anchor from cache ttl=23h59m59.931830254s 2026-10-30T21:27:26.148897Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:27:34.093273Z info Readiness succeeded in 8.249421749s 2026-10-30T21:27:34.093803Z info Envoy proxy is ready 2026-10-30T21:27:34.257747Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:28:30.247704Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:28:51.590285Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:28:56.176446Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T21:29:08.730Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 66 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "f153ad2f-f4af-43f4-ae47-0564362d0036" "maas.apps.e5d729db39642ea3674f.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.243:443 10.132.0.18:54704 - v1-completions-path [2026-10-30T21:29:08.997Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 56 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "ba66d146-5cda-4884-bff8-79afb6e629fd" "maas.apps.e5d729db39642ea3674f.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.243:443 10.132.0.18:54718 - v1-completions-path 2026-10-30T21:29:22.873422Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:29:39.777825Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:30:10.351584Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:30:21.138834Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:30:27.474804Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:30:38.315662Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T21:30:38.315703Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T21:30:38.315718Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T21:30:38.315722Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T21:30:38.193Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 318 122 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "f95f2829-ed9d-4d26-bc7e-4f24faf43e10" "maas.apps.e5d729db39642ea3674f.prod.konflux-ocp-ci.dev" "10.132.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.243:43436 10.134.0.243:443 10.132.0.18:53530 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-30T21:30:40.604Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 71 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "34e77e7d-1f17-424a-ba2e-7bac04f9e7a0" "maas.apps.e5d729db39642ea3674f.prod.konflux-ocp-ci.dev" "10.134.0.33:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.243:43780 10.134.0.243:443 10.132.0.18:53548 - v1-completions-path 2026-10-30T21:30:56.069868Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:31:00.746311Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T21:31:10.579Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 74 292 77 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "88ce6efc-784b-46dc-8dbc-864a786c2ba4" "maas.apps.e5d729db39642ea3674f.prod.konflux-ocp-ci.dev" "10.134.0.32:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.134.0.243:50372 10.134.0.243:443 10.132.0.18:37726 - v1-completions-path 2026-10-30T21:31:24.252184Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:31:30.694739Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:31:56.648469Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:32:15.811252Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:32:28.817367Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-30T21:32:52.853Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 34 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "78cfaac2-00e7-4df7-86d7-797c109209b0" "maas.apps.e5d729db39642ea3674f.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.243:443 10.132.0.18:57668 - v1-completions-path 2026-10-30T21:33:01.670749Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:33:23.396987Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:33:34.112803Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:35:52.052304Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-10-30T21:35:52.052384Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T21:35:52.052408Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T21:35:52.052423Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-30T21:35:52.052428Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-10-30T21:35:51.938Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 80 321 113 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "19a3dc0e-5cd8-4469-818b-fd2b649eac52" "maas.apps.e5d729db39642ea3674f.prod.konflux-ocp-ci.dev" "10.132.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.243:37604 10.134.0.243:443 10.132.0.18:34932 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T21:37:40.627046Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:37:44.366284Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T21:37:44.366317Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T21:37:44.366329Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T21:37:44.366334Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T21:37:44.251Z] "DELETE /v1/api-keys/e2e-trlp-test-key HTTP/1.1" 404 - via_upstream - "-" 0 29 115 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "a8f02dd9-ed20-4108-9eee-0fed3a6aa3b0" "maas.apps.e5d729db39642ea3674f.prod.konflux-ocp-ci.dev" "10.132.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.243:36556 10.134.0.243:443 10.132.0.18:56136 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T21:37:58.025277Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:38:02.516910Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:38:27.229395Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:40:26.271739Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T21:40:26.271774Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T21:40:26.271815Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-30T21:40:26.271823Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-30T21:40:26.206Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 65 - "34.228.250.194,10.132.0.18" "python-requests/2.34.2" "654c7100-e31d-4f7f-b32d-824caa7dc12f" "maas.apps.e5d729db39642ea3674f.prod.konflux-ocp-ci.dev" "10.132.0.51:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.243:34306 10.134.0.243:443 10.132.0.18:55548 - odh-ai-gateway-infra.maas-api-route.3 2026-10-30T21:40:35.064726Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:41:17.946622Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:41:43.616589Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:41:58.985069Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:42:11.870250Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:42:24.554342Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:42:29.454645Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:42:58.462754Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:46:34.203119Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:46:52.405780Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:47:13.590270Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:47:18.180636Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:47:34.433440Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-30T21:47:46.040495Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8