2026-11-03T09:37:07.010876Z info FLAG: --concurrency="0" 2026-11-03T09:37:07.010897Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-03T09:37:07.010901Z info FLAG: --help="false" 2026-11-03T09:37:07.010903Z info FLAG: --log_as_json="false" 2026-11-03T09:37:07.010905Z info FLAG: --log_caller="" 2026-11-03T09:37:07.010907Z info FLAG: --log_output_level="default:info" 2026-11-03T09:37:07.010909Z info FLAG: --log_stacktrace_level="default:none" 2026-11-03T09:37:07.010914Z info FLAG: --log_target="[stdout]" 2026-11-03T09:37:07.010916Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-03T09:37:07.010918Z info FLAG: --outlierLogPath="" 2026-11-03T09:37:07.010920Z info FLAG: --profiling="true" 2026-11-03T09:37:07.010921Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-03T09:37:07.010923Z info FLAG: --proxyLogLevel="warning" 2026-11-03T09:37:07.010925Z info FLAG: --serviceCluster="istio-proxy" 2026-11-03T09:37:07.010929Z info FLAG: --stsPort="0" 2026-11-03T09:37:07.010930Z info FLAG: --templateFile="" 2026-11-03T09:37:07.010932Z info FLAG: --tokenManagerPlugin="" 2026-11-03T09:37:07.010935Z info FLAG: --vklog="0" 2026-11-03T09:37:07.010937Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-03T09:37:07.011074Z info Proxy role ips=[10.132.0.218] type=router id=maas-default-gateway-openshift-default-c9ffffc7f-lkg9f.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-03T09:37:07.011114Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-03T09:37:07.012837Z info cpu limit detected as 2, setting concurrency 2026-11-03T09:37:07.013046Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-03T09:37:07.013058Z info JWT policy is third-party-jwt 2026-11-03T09:37:07.013061Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-03T09:37:07.014817Z info Starting default Istio SDS Server 2026-11-03T09:37:07.014838Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-03T09:37:07.014855Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-03T09:37:07.014992Z info Opening status port 15020 2026-11-03T09:37:07.016127Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-03T09:37:07.017470Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-03T09:37:07.018264Z info Starting proxy agent 2026-11-03T09:37:07.018294Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-03T09:37:07.026599Z info sds Starting SDS grpc server 2026-11-03T09:37:07.026623Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-03T09:37:07.088956Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-03T09:37:07.162632Z info cache generated new workload certificate resourceName=default latency=146.179162ms ttl=23h59m59.837374079s 2026-11-03T09:37:07.162677Z info cache Root cert has changed, start rotating root cert 2026-11-03T09:37:07.162720Z info cache returned workload trust anchor from cache ttl=23h59m59.837281252s 2026-11-03T09:37:07.309019Z info ads ADS: new connection for node:1 2026-11-03T09:37:07.315858Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-03T09:37:07.315908Z info cache read certificate from file resource=file-root:system 2026-11-03T09:37:07.343455Z info ads ADS: new connection for node:2 2026-11-03T09:37:07.343566Z info cache returned workload certificate from cache ttl=23h59m59.656435569s 2026-11-03T09:37:07.344541Z info ads ADS: new connection for node:3 2026-11-03T09:37:07.344695Z info cache returned workload trust anchor from cache ttl=23h59m59.655306643s 2026-11-03T09:37:07.465416Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T09:37:15.752873Z info Readiness succeeded in 8.744454589s 2026-11-03T09:37:15.753522Z info Envoy proxy is ready 2026-11-03T09:37:34.320888Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T09:38:52.558629Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T09:39:05.029Z] "POST /llm/e2e-dft-717214/anything/e2e-dft-717214-297b44 HTTP/1.1" 401 - - "-" 109 88 150 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "ab863eb5-871d-44c0-8249-5154c4603577" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-717214.llm.svc.cluster.local - 10.132.0.218:443 10.132.0.15:52472 - llm.external-model-e2e-dft-717214-cfe24eeace30.2 2026-11-03T09:39:36.605286Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T09:40:13.046Z] "POST /llm/e2e-dft-c2b995/anything/e2e-dft-c2b995-cd82c5 HTTP/1.1" 503 - - "-" 109 0 106 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "84f2fee4-2eb1-4844-8dfc-1ca72dc04700" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-c2b995.llm.svc.cluster.local - 10.132.0.218:443 10.132.0.15:40258 - llm.external-model-e2e-dft-c2b995-cc108e3e05b7.2 [2026-11-03T09:42:26.116Z] "POST /llm/e2e-dft-c2b995/anything/e2e-dft-c2b995-cd82c5 HTTP/1.1" 503 - - "-" 109 0 74 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "25846fdf-e405-4167-baff-e6fec8c24021" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-c2b995.llm.svc.cluster.local - 10.132.0.218:443 10.132.0.15:46598 - llm.external-model-e2e-dft-c2b995-cc108e3e05b7.2 [2026-11-03T09:42:58.038Z] "POST /llm/e2e-dft-c2b995/anything/e2e-dft-c2b995-cd82c5 HTTP/1.1" 503 - - "-" 109 0 70 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "b8243b08-e71c-4ff3-9c58-1fbc9a40d351" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-c2b995.llm.svc.cluster.local - 10.132.0.218:443 10.132.0.15:33452 - llm.external-model-e2e-dft-c2b995-cc108e3e05b7.2 [2026-11-03T09:43:08.745Z] "POST /llm/e2e-dft-c2b995/anything/e2e-dft-c2b995-cd82c5 HTTP/1.1" 503 - - "-" 109 0 69 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "8d046ffe-4bae-40cc-b7e3-f1ff2165835a" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-c2b995.llm.svc.cluster.local - 10.132.0.218:443 10.132.0.15:52276 - llm.external-model-e2e-dft-c2b995-cc108e3e05b7.2 2026-11-03T09:43:19.159843Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T09:44:20.781812Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T09:44:29.432706Z info ads ADS: "" 1 terminated 2026-11-03T09:44:32.650141Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T09:44:32.694274Z info ads ADS: new connection for node:4 2026-11-03T09:44:32.697562Z info cache read certificate from file resource=file-root:system 2026-11-03T09:44:36.773026Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T09:44:38.155145Z info ads ADS: "" 4 terminated 2026-11-03T09:44:44.227855Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T09:44:45.003290Z info ads ADS: new connection for node:5 2026-11-03T09:44:45.008576Z info cache read certificate from file resource=file-root:system 2026-11-03T09:44:46.294059Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T09:44:52.994137Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T09:45:05.475408Z info ads ADS: "" 5 terminated 2026-11-03T09:45:08.035595Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T09:45:31.983436Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T09:46:11.878574Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T09:47:38.027343Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T09:59:27.802116Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T09:59:56.154570Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:00:05.821374Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=26 2026-11-03T10:00:05.931436Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:00:05.931470Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:00:05.931522Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:00:05.931529Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-03T10:00:05.812Z] "POST /v1/api-keys HTTP/1.1" 400 - via_upstream - "-" 80 94 119 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "ff70cb88-a42f-4add-9914-469356a9ce59" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.134.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.218:34944 10.132.0.218:443 10.132.0.15:47336 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T10:00:31.693356Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:00:33.641526Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:00:33.641568Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:00:33.641584Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T10:00:33.641589Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T10:00:32.434Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 61 303 1207 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "012f13d3-2800-4323-8cb7-a8554ad472bb" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.134.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.218:46536 10.132.0.218:443 10.132.0.15:33138 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T10:00:34.149Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 2817 3089 81 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "b5e4364d-7995-4a95-b9af-9d60b0a7736d" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.132.0.218:59706 10.132.0.218:443 10.132.0.15:33146 - v1-chat-completions-path [2026-11-03T10:00:36.378Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 140 405 67 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "94017f3d-91a7-4445-84ce-fa1b6d1284bc" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.134.0.40:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.132.0.218:59718 10.132.0.218:443 10.132.0.15:33162 - v1-chat-completions-path 2026-11-03T10:00:50.102171Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:00:54.879961Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:00:56.882883Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:00:56.882921Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:00:56.882935Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T10:00:56.882940Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T10:00:55.672Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 324 1210 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "4c7d0455-60c9-4668-97b8-7d2bc228a95d" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.134.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.218:46536 10.132.0.218:443 10.132.0.15:46122 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T10:00:57.946993Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=25 [2026-11-03T10:00:57.872Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 91 3616 74 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "93c14353-358e-44e3-aa67-68d226ebdf9b" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.132.0.53:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.218:37738 10.132.0.218:443 10.132.0.15:46126 - v1-catch-all-model-routing 2026-11-03T10:01:04.252318Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T10:01:10.562Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 429 - - "-" 61 18 77 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "5a2a9141-d668-4fe8-8a0b-dbeaf9fe6871" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.218:443 10.132.0.15:55164 - v1-catch-all-path 2026-11-03T10:01:18.921144Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:01:28.520555Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:01:45.233739Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:01:56.996689Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:01:59.584448Z info ads ADS: new connection for node:6 2026-11-03T10:01:59.584654Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T10:01:59.584686Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-03T10:02:03.797467Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:02:52.784042Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:03:26.872501Z info ads ADS: "" 6 terminated with stream closed 2026-11-03T10:03:27.172438Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:03:32.010990Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:03:48.939453Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:06:58.295889Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:06:58.295924Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:06:58.295936Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:06:58.295941Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-03T10:06:58.239Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 56 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "5a9c08df-399d-4e5f-8708-04f6701e1b6b" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.134.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.218:49926 10.132.0.218:443 10.132.0.15:46388 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T10:07:09.430507Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:07:12.190512Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-03T10:07:41.331157Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:08:10.753931Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:08:26.380269Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:08:43.255321Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:08:55.700278Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:09:00.910127Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T10:09:04.252Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 83 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "5dee9f17-57eb-4f81-b5d8-04e556deb40c" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.218:47876 10.132.0.218:443 10.132.0.15:49980 - v1-completions-path [2026-11-03T10:09:04.810Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 68 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "79026654-8c68-47f5-8b84-a0e46d1043cc" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.218:47876 10.132.0.218:443 10.132.0.15:49994 - v1-completions-path 2026-11-03T10:09:06.710796Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-03T10:09:06.715534Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:09:06.715573Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:09:06.715591Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T10:09:06.715598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-03T10:09:06.657Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 524 58 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "1b05cb16-bca1-4df5-9d33-72f318556793" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.134.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.218:60844 10.132.0.218:443 10.132.0.15:49998 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T10:09:20.450270Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:10:18.443189Z info ads ADS: new connection for node:7 2026-11-03T10:10:18.448559Z info cache read certificate from file resource=file-root:system 2026-11-03T10:10:21.940911Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:10:24.672056Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:10:32.109636Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"582","StreamId":"12654425872474460389"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=25 [2026-11-03T10:10:32.044Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 65 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "2eebee1e-f0b2-4ed9-9d1d-331ecdedb832" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.132.0.218:443 10.132.0.15:49718 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-11-03T10:10:45.832760Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:10:45.832795Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:10:45.832805Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:10:45.832809Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T10:10:45.832966Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T10:10:45.712Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1708 120 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "dc91323c-7fb7-4bba-b546-4605f1458f8d" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.218:38222 10.132.0.218:443 10.132.0.15:53748 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T10:10:52.799461Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:10:52.799523Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:10:52.799533Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:10:52.799537Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-03T10:10:52.799695Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-11-03T10:10:52.667Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1709 132 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "f1720718-dd47-40f0-baae-144016b10ad2" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.218:33364 10.132.0.218:443 10.132.0.15:49468 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T10:10:55.118445Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:10:55.118528Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:10:55.118544Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:10:55.118549Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-03T10:10:55.118733Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-11-03T10:10:55.008Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1702 110 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "8dfb399f-099a-4efa-a74f-f4c8d276c2d0" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.218:33364 10.132.0.218:443 10.132.0.15:49500 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T10:11:36.918521Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:11:36.918559Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:11:36.918578Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T10:11:36.918584Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:11:36.918807Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T10:11:36.763Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1706 155 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "4c419f9f-f4c7-499b-9ce5-ca4404bf38b9" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.218:38222 10.132.0.218:443 10.132.0.15:47006 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T10:11:39.224810Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:11:39.224873Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:11:39.224884Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:11:39.224888Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-03T10:11:39.225049Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=26 [2026-11-03T10:11:39.114Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1707 110 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "eb8eb79e-bfe2-47b0-86b8-46bbbd911cea" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.218:33364 10.132.0.218:443 10.132.0.15:47014 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T10:11:55.521559Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:11:55.521592Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:11:55.521602Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-03T10:11:55.521606Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-03T10:11:55.521791Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-03T10:11:55.405Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1700 116 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "1087c806-861c-4be5-8e0a-af99b17544a6" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.132.0.218:38222 10.132.0.218:443 10.132.0.15:37646 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-03T10:12:04.343320Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:12:04.343366Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:12:04.343390Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-03T10:12:04.343397Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 [2026-11-03T10:12:04.228Z] "DELETE /v1/api-keys/581592d6-e667-4609-aced-fccf9cc6bd7f HTTP/1.1" 200 - via_upstream - "-" 0 359 115 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "ab4b37d7-2834-417e-92ec-14ac88e3466f" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.134.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.218:50350 10.132.0.218:443 10.132.0.15:54428 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T10:12:17.683125Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:12:17.702289Z info ads ADS: "" 7 terminated with stream closed 2026-11-03T10:12:20.448174Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:12:31.124021Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T10:12:34.333Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 61 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "85ebcaeb-eb69-4b14-baa2-0af583a215b2" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.218:47876 10.132.0.218:443 10.132.0.15:60724 - v1-completions-path 2026-11-03T10:12:49.562274Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:12:54.303626Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T10:12:57.270Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 288 62 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "518a86b2-ded4-4945-9b71-b32c3385d628" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.218:47876 10.132.0.218:443 10.132.0.15:53028 - v1-completions-path 2026-11-03T10:13:01.274406Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:13:01.274464Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:13:01.274505Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:13:01.274513Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-03T10:13:01.206Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 170 67 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "24be4bd0-7f6b-4a55-8cb5-fcff2def27c6" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.218:47876 10.132.0.218:443 10.132.0.15:35076 - v1-catch-all-path 2026-11-03T10:13:15.033362Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:13:41.825278Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T10:13:49.152Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 61 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "db4e5558-f78f-4792-a7e2-97a31dcecd61" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.218:47876 10.132.0.218:443 10.132.0.15:50726 - v1-completions-path [2026-11-03T10:13:49.515Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 64 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "0e149594-b5d6-90b2-88b9-653a2cb0cefa" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.218:39602 10.132.0.218:443 10.132.0.15:50730 - v1-completions-path 2026-11-03T10:14:22.615031Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:14:33.344453Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:14:40.210917Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:14:53.399854Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:14:53.399892Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:14:53.399908Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 2026-11-03T10:14:53.399912Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 [2026-11-03T10:14:53.288Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 317 111 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "72a50840-bfa2-4009-8462-b99a9152d332" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.134.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.218:34622 10.132.0.218:443 10.132.0.15:60170 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T10:14:53.857Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 60 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "6cc2055a-75f1-46a7-b670-b1e526c331a6" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.218:39602 10.132.0.218:443 10.132.0.15:60186 - v1-completions-path [2026-11-03T10:14:55.742Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 58 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "f1c3eedf-862d-491b-8694-4499e9ad83ff" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.218:39602 10.132.0.218:443 10.132.0.15:60206 - v1-completions-path [2026-11-03T10:14:56.510Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 59 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "3e5bd6d6-5cb5-42a7-bb0d-22d50526ba90" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.218:47876 10.132.0.218:443 10.132.0.15:60222 - v1-completions-path 2026-11-03T10:15:11.269723Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:15:16.006671Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:15:28.149966Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:15:38.537931Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:16:05.112454Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:16:15.985775Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:16:33.973711Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:17:03.680673Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T10:17:11.447Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 65 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "874d3bc1-e192-408b-8e48-b0edaef0bf67" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.218:39602 10.132.0.218:443 10.132.0.15:45680 - v1-completions-path 2026-11-03T10:17:25.374208Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:17:36.050762Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T10:20:09.528Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 47 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "967a0ddd-e2cb-49c7-863b-ff0da3743cac" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.218:443 10.132.0.15:50034 - v1-completions-path 2026-11-03T10:21:20.833283Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T10:21:20.833311Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T10:21:20.833315Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-11-03T10:21:20.833319Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-11-03T10:21:20.787Z] "POST /llm/e2e-trlp-test-simulated/v1/completions HTTP/1.1" 403 - - "-" 73 113 46 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "21e15949-d9e4-463a-af2d-35ed44814943" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-trlp-test-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.218:443 10.132.0.15:44774 - v1-completions-path 2026-11-03T10:21:42.940922Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:22:00.418914Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:22:05.141459Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:22:18.678379Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:22:38.332397Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:22:43.299831Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T10:22:46.972Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 64 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "f0ec718d-156f-4134-adec-38a93c65ceec" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.218:39602 10.132.0.218:443 10.132.0.15:60152 - v1-completions-path 2026-11-03T10:23:03.263817Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:23:13.944561Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:23:20.349906Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:23:32.369644Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:23:32.369684Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:23:32.369698Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:23:32.369704Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-03T10:23:32.241Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 67 310 128 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "2e276919-a026-41ca-be69-1e371eff7944" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.134.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.218:56406 10.132.0.218:443 10.132.0.15:47096 - odh-ai-gateway-infra.maas-api-route.3 2026-11-03T10:23:49.357164Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:23:55.858327Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:24:01.857713Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:24:10.017221Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:24:10.017268Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:24:10.017289Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-03T10:24:10.017296Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-03T10:24:09.904Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 66 309 113 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "c3b2b467-d68f-4519-b363-509cd9d47675" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.134.0.32:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.132.0.218:56406 10.132.0.218:443 10.132.0.15:42190 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-03T10:24:16.119Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 65 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "533cddc2-10b6-4af7-acdf-67f8a708e691" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.218:39602 10.132.0.218:443 10.132.0.15:55660 - v1-completions-path [2026-11-03T10:24:17.568Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 63 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "ffb3089b-44fa-403e-8211-224cdf493687" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.218:39602 10.132.0.218:443 10.132.0.15:55670 - v1-completions-path [2026-11-03T10:24:19.019Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 66 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "69978a3d-ced5-4642-adcd-442eb2d17358" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.218:39602 10.132.0.218:443 10.132.0.15:55676 - v1-completions-path 2026-11-03T10:24:34.275006Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:24:44.840416Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:24:51.477707Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:24:57.504355Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:25:09.401574Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-03T10:25:16.597Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 63 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "97cb54b3-2d96-4993-8f52-4bd1e0c71d2f" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "10.132.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.218:47876 10.132.0.218:443 10.132.0.15:40188 - v1-completions-path [2026-11-03T10:25:18.870Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 57 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "6bc8ce63-b613-4f2e-87b0-e3a358f31ec0" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.218:443 10.132.0.15:40190 - v1-completions-path 2026-11-03T10:25:33.101023Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:25:43.592569Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:25:48.498630Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:26:06.820368Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:26:12.993283Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:26:22.327261Z info cache event for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt : REMOVE, pushing to proxy 2026-11-03T10:26:36.599756Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:26:59.066612Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:27:20.681009Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:27:40.711284Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:27:51.533387Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:28:02.184504Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:28:12.978176Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:28:23.842134Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:28:34.406249Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-03T10:28:41.353133Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=26 2026-11-03T10:28:41.353161Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=26 2026-11-03T10:28:41.353164Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=26 2026-11-03T10:28:41.353168Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=26 [2026-11-03T10:28:41.319Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 401 - - "-" 66 88 34 - "44.212.242.249,10.132.0.15" "python-requests/2.34.2" "c52ff6d9-0010-4d27-bed8-1d95220f6545" "maas.apps.7fee51e1a8dedf3477d2.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.218:443 10.132.0.15:38808 - v1-completions-path