2026-11-03T21:24:44.324127Z info FLAG: --concurrency="0" 2026-11-03T21:24:44.324149Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T21:24:44.324153Z info FLAG: --help="false" 2026-11-03T21:24:44.324155Z info FLAG: --log_as_json="false" 2026-11-03T21:24:44.324157Z info FLAG: --log_caller="" 2026-11-03T21:24:44.324159Z info FLAG: --log_output_level="default:info" 2026-11-03T21:24:44.324161Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T21:24:44.324165Z info FLAG: --log_target="[stdout]" 2026-11-03T21:24:44.324167Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T21:24:44.324169Z info FLAG: --outlierLogPath="" 2026-11-03T21:24:44.324171Z info FLAG: --profiling="true" 2026-11-03T21:24:44.324173Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T21:24:44.324175Z info FLAG: --proxyLogLevel="warning" 2026-11-03T21:24:44.324177Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T21:24:44.324179Z info FLAG: --stsPort="0" 2026-11-03T21:24:44.324180Z info FLAG: --templateFile="" 2026-11-03T21:24:44.324182Z info FLAG: --tokenManagerPlugin="" 2026-11-03T21:24:44.324184Z info FLAG: --vklog="0" 2026-11-03T21:24:44.324187Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T21:24:44.324322Z info Proxy role ips=[10.134.0.45] type=router id=maas-default-gateway-openshift-default-778b7b5c47-b4xql.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T21:24:44.324371Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T21:24:44.325759Z info cpu limit detected as 2, setting concurrency 2026-11-03T21:24:44.325980Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T21:24:44.325991Z info JWT policy is third-party-jwt 2026-11-03T21:24:44.325994Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T21:24:44.327535Z info Starting default Istio SDS Server 2026-11-03T21:24:44.327555Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T21:24:44.327570Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T21:24:44.327938Z info Opening status port 15020 2026-11-03T21:24:44.328646Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T21:24:44.329220Z info sds Starting SDS grpc server 2026-11-03T21:24:44.329235Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T21:24:44.329911Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T21:24:44.330658Z info Starting proxy agent 2026-11-03T21:24:44.330687Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T21:24:44.392719Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T21:24:44.434590Z info cache generated new workload certificate resourceName=default latency=105.254322ms ttl=23h59m59.565412577s 2026-11-03T21:24:44.434634Z info cache Root cert has changed, start rotating root cert 2026-11-03T21:24:44.434661Z info cache returned workload trust anchor from cache ttl=23h59m59.565339446s 2026-11-03T21:24:44.531970Z info ads ADS: new connection for node:1 2026-11-03T21:24:44.532320Z info cache returned workload certificate from cache ttl=23h59m59.467683625s 2026-11-03T21:24:44.532333Z info ads ADS: new connection for node:2 2026-11-03T21:24:44.532579Z info cache returned workload trust anchor from cache ttl=23h59m59.467422744s 2026-11-03T21:24:44.607429Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:24:52.146494Z info Readiness succeeded in 7.824598987s 2026-11-03T21:24:52.146941Z info Envoy proxy is ready [2026-11-03T21:25:03.616Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 368 79 - "52.71.36.72,10.132.0.10" "curl/7.76.1" "632de44a-dbe1-4807-b87f-f81dcc592891" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "10.132.0.50:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.45:48348 10.134.0.45:443 10.132.0.10:43628 - v1-model-routing [2026-11-03T21:25:04.183Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 122 390 54 - "52.71.36.72,10.132.0.10" "curl/7.76.1" "e128b189-2fbf-4ef7-ba68-ccca1af9cbac" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "10.132.0.50:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.45:48348 10.134.0.45:443 10.132.0.10:43644 - v1-model-routing [2026-11-03T21:25:04.749Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 55 - "52.71.36.72,10.132.0.10" "curl/7.76.1" "209b58b9-203d-4ba1-ac95-4a523e287843" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.45:443 10.132.0.10:43650 - v1-model-routing [2026-11-03T21:25:05.019Z] "POST /v1/chat/completions HTTP/1.1" 429 - - "-" 122 18 54 - "52.71.36.72,10.132.0.10" "curl/7.76.1" "ad07628b-8ec2-45e3-81d4-80f8064d8d13" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.45:443 10.132.0.10:43656 - v1-model-routing 2026-11-03T21:25:06.118710Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T21:25:06.118733Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T21:25:06.118735Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T21:25:06.118739Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-11-03T21:25:06.098Z] "POST /v1/chat/completions HTTP/1.1" 401 - - "-" 122 88 20 - "52.71.36.72,10.132.0.10" "curl/7.76.1" "b8e55624-8ddd-4763-998f-7e143a309306" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.45:443 10.132.0.10:43658 - v1-model-routing 2026-11-03T21:30:41.108125Z info ads ADS: new connection for node:3 2026-11-03T21:30:41.111986Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-03T21:30:41.112031Z info cache read certificate from file resource=file-root:system 2026-11-03T21:30:45.263803Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:30:50.822482Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:31:05.507825Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:31:09.295696Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:31:09.295724Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:31:09.295732Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:31:09.295736Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T21:31:09.295898Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T21:31:09.164Z] "POST /anything/e2e-rt-sw-2e9b62-client HTTP/1.1" 200 - via_upstream - "-" 124 1586 131 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "d5c83501-bbaa-4d0b-8277-b2429a16b37b" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.45:53614 10.134.0.45:443 10.132.0.10:51100 - llm.external-model-e2e-rt-sw-2e9b62-211dd8de332c.2 2026-11-03T21:31:14.897753Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:31:14.897781Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:31:14.897789Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T21:31:14.897793Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:31:14.897939Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T21:31:14.781Z] "POST /anything/e2e-rt-sw-2e9b62-client HTTP/1.1" 200 - via_upstream - "-" 124 1578 116 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "ffac3c32-a9f5-414a-84b3-3230ecbbe541" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.0.45:53628 10.134.0.45:443 10.132.0.10:59774 - llm.external-model-e2e-rt-sw-2e9b62-211dd8de332c.0 2026-11-03T21:31:20.840145Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:31:38.798150Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:31:38.798182Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:31:38.798192Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:31:38.798197Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T21:31:38.798319Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-03T21:31:38.798347Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T21:31:38.735Z] "POST /llm/e2e-rt-sw-2e9b62/anything/e2e-rt-sw-2e9b62-client HTTP/1.1" 503 UH no_healthy_upstream - "-" 124 19 62 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "7019c32a-ab63-4e99-8be7-df36bcac9660" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rt-sw-2e9b62.llm.svc.cluster.local - 10.134.0.45:443 10.132.0.10:42516 - llm.external-model-e2e-rt-sw-2e9b62-211dd8de332c.2 2026-11-03T21:31:55.674390Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:31:55.674430Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:31:55.674444Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T21:31:55.674450Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:31:55.674652Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-11-03T21:31:55.608Z] "POST /llm/e2e-rt-sw-2e9b62/anything/e2e-rt-sw-2e9b62-client HTTP/1.1" 503 UH no_healthy_upstream - "-" 124 19 66 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "348eb4bc-4cae-46bf-9b9a-63cc4ec214a5" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rt-sw-2e9b62.llm.svc.cluster.local - 10.134.0.45:443 10.132.0.10:43924 - llm.external-model-e2e-rt-sw-2e9b62-211dd8de332c.2 2026-11-03T21:32:13.107729Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:32:25.684642Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:32:37.204072Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T21:33:06.894Z] "POST /llm/e2e-rt-eq-73b57e/anything/e2e-rt-eq-73b57e-client HTTP/1.1" 503 - - "-" 124 0 73 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "208d30be-c41d-4e96-a1e8-b56a28c677f0" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rt-eq-73b57e.llm.svc.cluster.local - 10.134.0.45:443 10.132.0.10:37072 - llm.external-model-e2e-rt-eq-73b57e-b255ad122fba.4 2026-11-03T21:33:30.298902Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:33:35.965564Z info ads ADS: "" 3 terminated with stream closed 2026-11-03T21:33:37.507213Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:33:51.754939Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:35:30.854141Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:35:45.862422Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:36:08.881041Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:37:39.595371Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:44:26.054066Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:44:26.054100Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:44:26.054112Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T21:44:26.054117Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-03T21:44:25.946Z] "DELETE /v1/api-keys/0ba56fc5-cd52-4136-a57f-069aae815fe7 HTTP/1.1" 200 - via_upstream - "-" 0 321 107 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "efc07008-356b-4b5a-a3a4-7595c7e4bc32" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "10.133.0.27:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.45:40602 10.134.0.45:443 10.132.0.10:56980 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:44:28.333118Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:44:28.333151Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:44:28.333165Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:44:28.333169Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T21:44:28.227Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 105 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "766b66a9-e8a2-463e-bead-f653bfbe70c7" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "10.133.0.27:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.45:47956 10.134.0.45:443 10.132.0.10:56984 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:44:29.142817Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:44:29.142848Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:44:29.142861Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T21:44:29.142865Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T21:44:29.039Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 102 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "3209d9cd-b017-4dc4-b728-cbdac76f8693" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "10.133.0.27:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.45:47956 10.134.0.45:443 10.132.0.10:57008 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T21:44:31.407Z] "POST /maas-api/v1/api-keys HTTP/1.1" 403 - - "-" 77 0 87 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "0725b8cf-991b-4cb2-b722-b172f7329e2d" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.134.0.45:443 10.132.0.10:49032 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T21:44:43.208Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 82 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "97cc0043-11f2-4db3-bad7-72b466dbae0a" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "10.132.0.50:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.45:39020 10.134.0.45:443 10.132.0.10:41448 - v1-completions-path [2026-11-03T21:44:51.819Z] "POST /maas-api/internal/v1/api-keys/validate HTTP/1.1" 404 NR route_not_found - "-" 31 0 6 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "b92efc29-3015-48fd-be3b-0e2dd12addc9" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "-" - - 10.134.0.45:443 10.132.0.10:59882 - - 2026-11-03T21:44:54.764854Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:44:54.764891Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:44:54.764907Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:44:54.764912Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T21:44:54.646Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 118 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "2265d37e-4d37-432e-accf-c3e4a0deed7e" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "10.133.0.27:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.45:47956 10.134.0.45:443 10.132.0.10:59892 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T21:44:58.058140Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:45:37.494771Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T21:56:35.225687Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-03T21:58:53.574690Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:58:53.574737Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:58:53.574756Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T21:58:53.574764Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T21:58:54.270887Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:58:54.270923Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T21:58:54.270936Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T21:58:54.270940Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T21:58:53.562Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 91 12 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "a63a3ce6-3d8c-4c8b-9088-1cd9a1dd2889" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "10.133.0.27:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.45:40884 10.134.0.45:443 10.132.0.10:46750 - odh-ai-gateway-infra.maas-api-route.4 [2026-11-03T21:58:54.106Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 322 164 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "41a2963d-35c5-4c32-a1e0-eaf2df880700" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "10.133.0.27:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.45:40886 10.134.0.45:443 10.132.0.10:46752 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T21:58:54.470Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 135 1811 118 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "68445f8f-653c-4576-842d-9e961dab6637" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "10.132.0.50:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.45:39020 10.134.0.45:443 10.132.0.10:46766 - v1-chat-completions-path 2026-11-03T22:00:13.624856Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:00:13.624893Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:00:13.624908Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T22:00:13.624913Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:00:13.939740Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:00:13.939770Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:00:13.939781Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-03T22:00:13.939786Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-03T22:00:14.201026Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-03T22:00:13.504Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 120 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "a30168c5-3fe2-4627-ba88-a89d1ebce35e" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "10.133.0.27:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.45:57526 10.134.0.45:443 10.132.0.10:56540 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T22:00:13.834Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 322 105 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "4b3d95e0-f0bb-4067-b69d-cc10c732a3ee" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "10.133.0.27:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.45:57536 10.134.0.45:443 10.132.0.10:56550 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T22:00:14.138Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 136 332 63 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "71806c18-1b33-47d0-a477-7a16fca40b65" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "10.132.0.50:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.45:48348 10.134.0.45:443 10.132.0.10:56556 - v1-chat-completions-path [2026-11-03T22:03:58.213Z] "POST /maas-api/v1/tokens HTTP/1.1" 401 - - "-" 20 88 22 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "5c84d882-0e3d-934a-82b5-3f8811d72e56" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.134.0.45:443 10.132.0.10:47388 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T22:04:48.600781Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:04:48.600826Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:04:48.600849Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T22:04:48.600855Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-03T22:04:48.482Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 567 118 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "b845ee97-b80e-45e8-b04f-0383b64f713c" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "10.133.0.27:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.45:50934 10.134.0.45:443 10.132.0.10:52944 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T22:12:27.814Z] "GET /v1/tenants HTTP/1.1" 404 NR route_not_found - "-" 0 0 7 - "52.71.36.72,10.132.0.10" "python-requests/2.34.2" "57a5f50a-e4c0-4ead-91c7-c90b8d2def9b" "maas.apps.fbd0eaee6215b2f00703.prod.konflux-ocp-ci.dev" "-" - - 10.134.0.45:443 10.132.0.10:59660 - -