2026-10-29T21:53:28.405696Z info FLAG: --concurrency="0" 2026-10-29T21:53:28.405714Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-29T21:53:28.405718Z info FLAG: --help="false" 2026-10-29T21:53:28.405720Z info FLAG: --log_as_json="false" 2026-10-29T21:53:28.405722Z info FLAG: --log_caller="" 2026-10-29T21:53:28.405724Z info FLAG: --log_output_level="default:info" 2026-10-29T21:53:28.405726Z info FLAG: --log_stacktrace_level="default:none" 2026-10-29T21:53:28.405731Z info FLAG: --log_target="[stdout]" 2026-10-29T21:53:28.405733Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-29T21:53:28.405735Z info FLAG: --outlierLogPath="" 2026-10-29T21:53:28.405737Z info FLAG: --profiling="true" 2026-10-29T21:53:28.405739Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-29T21:53:28.405741Z info FLAG: --proxyLogLevel="warning" 2026-10-29T21:53:28.405743Z info FLAG: --serviceCluster="istio-proxy" 2026-10-29T21:53:28.405745Z info FLAG: --stsPort="0" 2026-10-29T21:53:28.405747Z info FLAG: --templateFile="" 2026-10-29T21:53:28.405749Z info FLAG: --tokenManagerPlugin="" 2026-10-29T21:53:28.405751Z info FLAG: --vklog="0" 2026-10-29T21:53:28.405753Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-29T21:53:28.405901Z info Proxy role ips=[10.134.0.44] type=router id=maas-default-gateway-openshift-default-769df7846c-9xmcs.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-29T21:53:28.405951Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-29T21:53:28.407180Z info cpu limit detected as 2, setting concurrency 2026-10-29T21:53:28.407441Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-29T21:53:28.407452Z info JWT policy is third-party-jwt 2026-10-29T21:53:28.407455Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-29T21:53:28.409218Z info Starting default Istio SDS Server 2026-10-29T21:53:28.409240Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-29T21:53:28.409262Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-29T21:53:28.409534Z info Opening status port 15020 2026-10-29T21:53:28.410654Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-29T21:53:28.412698Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-29T21:53:28.413938Z info Starting proxy agent 2026-10-29T21:53:28.413978Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-29T21:53:28.421029Z info sds Starting SDS grpc server 2026-10-29T21:53:28.421053Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-29T21:53:28.458276Z info cache generated new workload certificate resourceName=default latency=47.911656ms ttl=23h59m59.541726904s 2026-10-29T21:53:28.458309Z info cache Root cert has changed, start rotating root cert 2026-10-29T21:53:28.458340Z info cache returned workload trust anchor from cache ttl=23h59m59.541660267s 2026-10-29T21:53:28.481078Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-29T21:53:28.604128Z info ads ADS: new connection for node:1 2026-10-29T21:53:28.604451Z info cache returned workload certificate from cache ttl=23h59m59.395552855s 2026-10-29T21:53:28.604476Z info ads ADS: new connection for node:2 2026-10-29T21:53:28.604769Z info cache returned workload trust anchor from cache ttl=23h59m59.39523307s 2026-10-29T21:53:28.656017Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:53:35.009731Z info Readiness succeeded in 6.607680031s 2026-10-29T21:53:35.010028Z info Envoy proxy is ready 2026-10-29T21:53:41.280616Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:53:43.214830Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:53:44.022017Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:53:45.822333Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:53:48.344178Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:53:51.371208Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:53:55.631327Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:53:58.373920Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:54:03.045161Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-29T21:54:31.591Z] "POST /v1/chat/completions HTTP/1.1" 401 - - "-" 122 0 21 - "52.71.36.72,10.132.0.13" "curl/7.76.1" "0252a002-2a93-4ea8-9c32-e387527018ea" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.44:443 10.132.0.13:38352 - v1-model-routing [2026-10-29T21:54:31.894Z] "POST /v1/chat/completions HTTP/1.1" 401 - - "-" 122 0 14 - "52.71.36.72,10.132.0.13" "curl/7.76.1" "61bec9bc-dda8-40b8-a782-ce1b20c12ca4" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.44:443 10.132.0.13:38372 - v1-model-routing 2026-10-29T21:54:37.326399Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim fd5039db79d000f31669de1fde37ea175d3ad05f2b14341d6936fb564cab5553: Failed to get response CEL value: Failed to retrieve gRPC response: Received response with size 0 thread=25 [2026-10-29T21:54:37.240Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 337 86 - "52.71.36.72,10.132.0.13" "curl/7.76.1" "f7a51c38-77d8-4f9a-92c9-b64e3e1084f5" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "10.134.0.48:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.44:55308 10.134.0.44:443 10.132.0.13:38390 - v1-model-routing 2026-10-29T21:54:37.997030Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-29T21:54:37.997053Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-29T21:54:37.997055Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-29T21:54:37.997059Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-10-29T21:54:37.936Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 87 60 - "52.71.36.72,10.132.0.13" "curl/7.76.1" "3d85f097-1449-45ea-94df-96f657d83e49" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.44:443 10.132.0.13:38392 - v1-model-routing 2026-10-29T22:03:39.962981Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim fd5039db79d000f31669de1fde37ea175d3ad05f2b14341d6936fb564cab5553: Task failed: "0" thread=24 2026-10-29T22:03:40.259852Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim fd5039db79d000f31669de1fde37ea175d3ad05f2b14341d6936fb564cab5553: Task failed: "0" thread=24 2026-10-29T22:03:40.362009Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim fd5039db79d000f31669de1fde37ea175d3ad05f2b14341d6936fb564cab5553: Task failed: "0" thread=25 [2026-10-29T22:03:39.866Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 96 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "5471a1f8-74df-4466-b4bb-41910f476e9f" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "10.133.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.44:54574 10.134.0.44:443 10.132.0.13:58336 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-29T22:03:40.172Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 87 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "c4c36519-2539-4735-8b9e-c1b1c67fe1df" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "10.133.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.44:54574 10.134.0.44:443 10.132.0.13:58342 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-29T22:03:40.279Z] "DELETE /v1/api-keys/dcd03c43-24f1-416a-93f4-04a2294ca38b HTTP/1.1" 200 - via_upstream - "-" 0 321 83 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "8e5b3b5b-1b2e-4ab8-bb7a-5722d81f801d" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "10.133.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.44:54590 10.134.0.44:443 10.132.0.13:58346 - odh-ai-gateway-infra.maas-api-route.3 2026-10-29T22:03:41.084539Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim fd5039db79d000f31669de1fde37ea175d3ad05f2b14341d6936fb564cab5553: Task failed: "0" thread=25 2026-10-29T22:03:41.176730Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim fd5039db79d000f31669de1fde37ea175d3ad05f2b14341d6936fb564cab5553: Task failed: "0" thread=25 [2026-10-29T22:03:41.005Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 78 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "30e201fa-51ee-4002-8b4d-92cba113978f" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "10.133.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.44:54590 10.134.0.44:443 10.132.0.13:58362 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-29T22:03:41.104Z] "DELETE /v1/api-keys/648158dc-1213-417a-be55-feef13a6998f HTTP/1.1" 200 - via_upstream - "-" 0 321 72 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "655c1f1b-ff7b-4017-98b7-91c2c742cefe" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "10.133.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.44:54590 10.134.0.44:443 10.132.0.13:58376 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-29T22:03:42.297Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 58 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "8aa3e927-f028-490b-8f36-afdebe302275" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.134.0.44:443 10.132.0.13:58392 - odh-ai-gateway-infra.maas-api-route.3 2026-10-29T22:03:43.445307Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim fd5039db79d000f31669de1fde37ea175d3ad05f2b14341d6936fb564cab5553: Task failed: "0" thread=24 [2026-10-29T22:03:43.366Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 79 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "506fbd20-aa9d-4c8e-8141-05a21bddb42b" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "10.133.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.44:54574 10.134.0.44:443 10.132.0.13:58414 - odh-ai-gateway-infra.maas-api-route.3 2026-10-29T22:03:51.774887Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim fd5039db79d000f31669de1fde37ea175d3ad05f2b14341d6936fb564cab5553: Task failed: "0" thread=24 [2026-10-29T22:03:51.696Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 317 78 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "e6de3bd4-008b-458f-b0f1-c844035830ad" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "10.133.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.44:54574 10.134.0.44:443 10.132.0.13:56206 - odh-ai-gateway-infra.maas-api-route.3 2026-10-29T22:03:57.172010Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-29T22:03:57.172035Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-29T22:03:57.172039Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-29T22:03:57.172054Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-10-29T22:03:57.140Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 76 77 31 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "76330dbc-aa8e-4526-9b92-4d14e102eab0" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.44:443 10.132.0.13:56220 - v1-completions-path 2026-10-29T22:03:58.026461Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim fd5039db79d000f31669de1fde37ea175d3ad05f2b14341d6936fb564cab5553: Failed to get response CEL value: Failed to retrieve gRPC response: Received response with size 0 thread=25 [2026-10-29T22:03:57.952Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 290 73 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "95708353-f05e-4b08-b75d-8b359d531471" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "10.134.0.48:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.44:55308 10.134.0.44:443 10.132.0.13:56228 - v1-completions-path [2026-10-29T22:03:58.406Z] "POST /maas-api/internal/v1/api-keys/cleanup HTTP/1.1" 404 NR route_not_found - "-" 2 0 11 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "1ff4ab81-fded-42f9-b35d-7cad84ba76c7" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "-" - - 10.134.0.44:443 10.132.0.13:56234 - - 2026-10-29T22:09:51.417861Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim fd5039db79d000f31669de1fde37ea175d3ad05f2b14341d6936fb564cab5553: Task failed: "0" thread=24 [2026-10-29T22:09:50.773Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 69 311 644 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "2c0b3545-3fe2-48f9-b5ed-400ac9fbddf4" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "10.133.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.44:38720 10.134.0.44:443 10.132.0.13:39038 - odh-ai-gateway-infra.maas-api-route.3 2026-10-29T22:09:52.121062Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim fd5039db79d000f31669de1fde37ea175d3ad05f2b14341d6936fb564cab5553: Failed to get response CEL value: Failed to retrieve gRPC response: Received response with size 0 thread=25 [2026-10-29T22:09:51.900Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 110 326 221 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "0d825ea5-c1d0-4504-a173-351c98862a36" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "10.134.0.48:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.44:55308 10.134.0.44:443 10.132.0.13:39050 - v1-chat-completions-path