2026-10-29T21:52:45.918850Z info FLAG: --concurrency="0" 2026-10-29T21:52:45.918868Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-29T21:52:45.918872Z info FLAG: --help="false" 2026-10-29T21:52:45.918874Z info FLAG: --log_as_json="false" 2026-10-29T21:52:45.918876Z info FLAG: --log_caller="" 2026-10-29T21:52:45.918878Z info FLAG: --log_output_level="default:info" 2026-10-29T21:52:45.918880Z info FLAG: --log_stacktrace_level="default:none" 2026-10-29T21:52:45.918885Z info FLAG: --log_target="[stdout]" 2026-10-29T21:52:45.918887Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-29T21:52:45.918889Z info FLAG: --outlierLogPath="" 2026-10-29T21:52:45.918891Z info FLAG: --profiling="true" 2026-10-29T21:52:45.918893Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-29T21:52:45.918895Z info FLAG: --proxyLogLevel="warning" 2026-10-29T21:52:45.918897Z info FLAG: --serviceCluster="istio-proxy" 2026-10-29T21:52:45.918899Z info FLAG: --stsPort="0" 2026-10-29T21:52:45.918901Z info FLAG: --templateFile="" 2026-10-29T21:52:45.918902Z info FLAG: --tokenManagerPlugin="" 2026-10-29T21:52:45.918905Z info FLAG: --vklog="0" 2026-10-29T21:52:45.918907Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-29T21:52:45.919109Z info Proxy role ips=[10.133.0.37] type=router id=maas-default-gateway-openshift-default-769df7846c-b4jg2.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-29T21:52:45.919169Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-29T21:52:45.920134Z info cpu limit detected as 2, setting concurrency 2026-10-29T21:52:45.920390Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-29T21:52:45.920401Z info JWT policy is third-party-jwt 2026-10-29T21:52:45.920404Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-29T21:52:45.922071Z info Starting default Istio SDS Server 2026-10-29T21:52:45.922107Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-29T21:52:45.922131Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-29T21:52:45.922496Z info Opening status port 15020 2026-10-29T21:52:45.923340Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-29T21:52:45.924039Z info sds Starting SDS grpc server 2026-10-29T21:52:45.924073Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-29T21:52:45.924707Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-29T21:52:45.925504Z info Starting proxy agent 2026-10-29T21:52:45.925533Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-29T21:52:45.991812Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-29T21:52:46.055621Z info cache generated new workload certificate resourceName=default latency=131.411457ms ttl=23h59m59.944382232s 2026-10-29T21:52:46.055653Z info cache Root cert has changed, start rotating root cert 2026-10-29T21:52:46.055680Z info cache returned workload trust anchor from cache ttl=23h59m59.944320108s 2026-10-29T21:52:46.099742Z info ads ADS: new connection for node:1 2026-10-29T21:52:46.099808Z info cache returned workload certificate from cache ttl=23h59m59.900193559s 2026-10-29T21:52:46.100172Z info ads ADS: new connection for node:2 2026-10-29T21:52:46.100244Z info cache returned workload trust anchor from cache ttl=23h59m59.899757738s 2026-10-29T21:52:46.148291Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:52:51.974938Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:52:52.161270Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:52:52.290451Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:52:52.420600Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:52:52.554788Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:52:52.690340Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:52:52.831742Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:52:52.965400Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:52:53.084761Z info Readiness succeeded in 7.168490913s 2026-10-29T21:52:53.085132Z info Envoy proxy is ready 2026-10-29T21:52:53.342119Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:52:54.069199Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:52:54.799677Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:52:55.586019Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:52:56.425393Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:52:57.183617Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:52:58.136259Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:52:59.083530Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:52:59.845289Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:53:00.638868Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:53:01.437289Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:53:02.340849Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:53:03.146552Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:53:03.959722Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:53:08.500168Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:53:09.433664Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:53:41.291036Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:53:43.224348Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:53:44.016514Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:53:45.789620Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:53:48.320950Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:53:51.371200Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:53:55.558714Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:53:58.302949Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-29T21:54:02.991171Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-29T21:54:31.487Z] "POST /v1/chat/completions HTTP/1.1" 401 - - "-" 122 0 18 - "52.71.36.72,10.132.0.13" "curl/7.76.1" "505c1c8d-221d-4de5-bb56-f0baf83b49d3" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.37:443 10.132.0.13:50630 - v1-model-routing 2026-10-29T21:54:32.090731Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim fd5039db79d000f31669de1fde37ea175d3ad05f2b14341d6936fb564cab5553: Task failed: "0" thread=25 [2026-10-29T21:54:31.641Z] "POST /v1/chat/completions HTTP/1.1" 401 - - "-" 122 0 12 - "52.71.36.72,10.132.0.13" "curl/7.76.1" "a339a882-0602-4ddd-ae8b-55f21c7e63c9" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.37:443 10.132.0.13:50636 - v1-model-routing [2026-10-29T21:54:31.767Z] "POST /v1/chat/completions HTTP/1.1" 401 - - "-" 122 0 17 - "52.71.36.72,10.132.0.13" "curl/7.76.1" "f7209797-907f-4400-b8f9-7343543ab90b" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.37:443 10.132.0.13:50648 - v1-model-routing [2026-10-29T21:54:31.997Z] "DELETE /v1/api-keys/c38c6720-a365-4891-8d60-f3bdab5e6909 HTTP/1.1" 200 - via_upstream - "-" 0 365 93 - "52.71.36.72,10.132.0.13" "curl/7.76.1" "8d9cf65d-198e-468e-ab2e-2c84b9d696b6" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "10.133.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.37:35882 10.133.0.37:443 10.132.0.13:50660 - odh-ai-gateway-infra.maas-api-route.3 2026-10-29T21:54:37.435224Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim fd5039db79d000f31669de1fde37ea175d3ad05f2b14341d6936fb564cab5553: Failed to get response CEL value: Failed to retrieve gRPC response: Received response with size 0 thread=25 [2026-10-29T21:54:37.351Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 461 83 - "52.71.36.72,10.132.0.13" "curl/7.76.1" "a4fb4506-b7f2-4930-8efa-9735ddeb9522" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "10.134.0.48:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.37:38772 10.133.0.37:443 10.132.0.13:50680 - v1-model-routing 2026-10-29T21:54:37.615707Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-29T21:54:37.615731Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-29T21:54:37.615734Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-29T21:54:37.615738Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-10-29T21:54:37.555Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 87 60 - "52.71.36.72,10.132.0.13" "curl/7.76.1" "7214b0a4-c8aa-40bc-80fe-3237850d0242" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.37:443 10.132.0.13:50682 - v1-model-routing 2026-10-29T21:54:38.701322Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim fd5039db79d000f31669de1fde37ea175d3ad05f2b14341d6936fb564cab5553: Task failed: "0" thread=25 [2026-10-29T21:54:38.693Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 20 7 - "52.71.36.72,10.132.0.13" "curl/7.76.1" "0187f5b5-7f16-4017-960a-9128b241ea04" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "10.133.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.37:35882 10.133.0.37:443 10.132.0.13:50688 - odh-ai-gateway-infra.maas-api-route.4 2026-10-29T22:03:39.072399Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim fd5039db79d000f31669de1fde37ea175d3ad05f2b14341d6936fb564cab5553: Task failed: "0" thread=24 2026-10-29T22:03:39.673011Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim fd5039db79d000f31669de1fde37ea175d3ad05f2b14341d6936fb564cab5553: Task failed: "0" thread=24 [2026-10-29T22:03:38.972Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 100 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "ba3b8ee3-d7b1-4279-9c91-4e7ae31df2cc" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "10.133.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.37:36440 10.133.0.37:443 10.132.0.13:55664 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-29T22:03:39.594Z] "DELETE /v1/api-keys/04c5e20b-bcfd-4813-9bf2-37ef9aaa17b5 HTTP/1.1" 200 - via_upstream - "-" 0 321 78 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "40068a94-db43-4439-9bbb-1b71e7acaac5" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "10.133.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.37:36440 10.133.0.37:443 10.132.0.13:55670 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-29T22:03:41.589Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 81 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "721107d8-45d7-4405-998d-b93c0be625dd" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.133.0.37:443 10.132.0.13:55674 - odh-ai-gateway-infra.maas-api-route.3 2026-10-29T22:03:42.739745Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim fd5039db79d000f31669de1fde37ea175d3ad05f2b14341d6936fb564cab5553: Task failed: "0" thread=24 [2026-10-29T22:03:42.646Z] "DELETE /v1/api-keys/00023319-f252-42ec-81a4-0c9cff883165 HTTP/1.1" 200 - via_upstream - "-" 0 321 93 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "32fc8d35-5026-43f9-90f9-7cb130a7d747" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "10.133.0.34:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.37:36440 10.133.0.37:443 10.132.0.13:55682 - odh-ai-gateway-infra.maas-api-route.3 2026-10-29T22:09:51.833180Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim fd5039db79d000f31669de1fde37ea175d3ad05f2b14341d6936fb564cab5553: Failed to get response CEL value: Failed to retrieve gRPC response: Received response with size 0 thread=24 [2026-10-29T22:09:51.467Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 110 326 365 - "52.71.36.72,10.132.0.13" "python-requests/2.34.2" "ab74e681-a228-4725-9f27-8412fb7fee67" "maas.apps.0a85c7e05f523a95363d.prod.konflux-ocp-ci.dev" "10.134.0.48:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.37:55632 10.133.0.37:443 10.132.0.13:52612 - v1-chat-completions-path