2026-10-31T00:38:39.441794Z info FLAG: --concurrency="0" 2026-10-31T00:38:39.441810Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-10-31T00:38:39.441815Z info FLAG: --help="false" 2026-10-31T00:38:39.441817Z info FLAG: --log_as_json="false" 2026-10-31T00:38:39.441819Z info FLAG: --log_caller="" 2026-10-31T00:38:39.441821Z info FLAG: --log_output_level="default:info" 2026-10-31T00:38:39.441822Z info FLAG: --log_stacktrace_level="default:none" 2026-10-31T00:38:39.441827Z info FLAG: --log_target="[stdout]" 2026-10-31T00:38:39.441829Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-10-31T00:38:39.441831Z info FLAG: --outlierLogPath="" 2026-10-31T00:38:39.441833Z info FLAG: --profiling="true" 2026-10-31T00:38:39.441835Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-10-31T00:38:39.441837Z info FLAG: --proxyLogLevel="warning" 2026-10-31T00:38:39.441838Z info FLAG: --serviceCluster="istio-proxy" 2026-10-31T00:38:39.441840Z info FLAG: --stsPort="0" 2026-10-31T00:38:39.441842Z info FLAG: --templateFile="" 2026-10-31T00:38:39.441844Z info FLAG: --tokenManagerPlugin="" 2026-10-31T00:38:39.441846Z info FLAG: --vklog="0" 2026-10-31T00:38:39.441849Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-10-31T00:38:39.442046Z info Proxy role ips=[10.133.0.36] type=router id=maas-default-gateway-openshift-default-6cc458d887-fk527.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-10-31T00:38:39.442090Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-10-31T00:38:39.443139Z info cpu limit detected as 2, setting concurrency 2026-10-31T00:38:39.443729Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-10-31T00:38:39.443739Z info JWT policy is third-party-jwt 2026-10-31T00:38:39.443742Z info using credential fetcher of JWT type in cluster.local trust domain 2026-10-31T00:38:39.445034Z info Starting default Istio SDS Server 2026-10-31T00:38:39.445057Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-10-31T00:38:39.445091Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-10-31T00:38:39.445556Z info Opening status port 15020 2026-10-31T00:38:39.446474Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-10-31T00:38:39.447118Z info sds Starting SDS grpc server 2026-10-31T00:38:39.447156Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-10-31T00:38:39.447969Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-10-31T00:38:39.448816Z info Starting proxy agent 2026-10-31T00:38:39.448847Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-10-31T00:38:39.511377Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-10-31T00:38:39.667366Z info cache generated new workload certificate resourceName=default latency=220.071555ms ttl=23h59m59.332638223s 2026-10-31T00:38:39.667405Z info cache Root cert has changed, start rotating root cert 2026-10-31T00:38:39.667441Z info cache returned workload trust anchor from cache ttl=23h59m59.332559405s 2026-10-31T00:38:39.710875Z info ads ADS: new connection for node:1 2026-10-31T00:38:39.711183Z info cache returned workload certificate from cache ttl=23h59m59.288819008s 2026-10-31T00:38:39.711506Z info ads ADS: new connection for node:2 2026-10-31T00:38:39.711876Z info cache returned workload trust anchor from cache ttl=23h59m59.288125621s 2026-10-31T00:38:39.790410Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T00:38:47.490197Z info Readiness succeeded in 8.050976104s 2026-10-31T00:38:47.490523Z info Envoy proxy is ready 2026-10-31T00:44:23.320364Z info ads ADS: new connection for node:3 2026-10-31T00:44:23.324007Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-10-31T00:44:23.324049Z info cache read certificate from file resource=file-root:system 2026-10-31T00:44:28.954663Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T00:44:32.345431Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T00:44:44.337345Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T00:44:48.761020Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T00:44:48.761059Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T00:44:48.761073Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T00:44:48.761079Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T00:44:48.761265Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-31T00:44:48.653Z] "POST /llm/e2e-rt-sw-c292bd/anything/e2e-rt-sw-c292bd-client HTTP/1.1" 503 UH no_healthy_upstream - "-" 124 19 107 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "992ad77d-d3ab-4e01-8ae1-cf85f4189ad2" "maas.apps.f7fad04c82cf1a4da903.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rt-sw-c292bd.llm.svc.cluster.local - 10.133.0.36:443 10.132.0.11:50992 - llm.external-model-e2e-rt-sw-c292bd-ef50224c2876.2 2026-10-31T00:44:54.042283Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T00:44:54.042317Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T00:44:54.042326Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T00:44:54.042331Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T00:44:54.042484Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-31T00:44:53.962Z] "POST /llm/e2e-rt-sw-c292bd/anything/e2e-rt-sw-c292bd-client HTTP/1.1" 503 UH no_healthy_upstream - "-" 124 19 80 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "cc5ca537-5e72-4483-a2b0-13641fcf6d12" "maas.apps.f7fad04c82cf1a4da903.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rt-sw-c292bd.llm.svc.cluster.local - 10.133.0.36:443 10.132.0.11:50998 - llm.external-model-e2e-rt-sw-c292bd-ef50224c2876.2 2026-10-31T00:44:54.685516Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T00:45:04.650678Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T00:45:04.650742Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T00:45:04.650754Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T00:45:04.650759Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T00:45:04.650925Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 [2026-10-31T00:45:04.572Z] "POST /llm/e2e-rt-sw-c292bd/anything/e2e-rt-sw-c292bd-client HTTP/1.1" 503 UH no_healthy_upstream - "-" 124 19 78 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "a9c64ba9-7c43-4615-9cd5-b155e9851007" "maas.apps.f7fad04c82cf1a4da903.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rt-sw-c292bd.llm.svc.cluster.local - 10.133.0.36:443 10.132.0.11:52688 - llm.external-model-e2e-rt-sw-c292bd-ef50224c2876.2 2026-10-31T00:46:01.886019Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T00:46:41.953197Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T00:46:41.953228Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T00:46:41.953238Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T00:46:41.953243Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T00:46:41.953387Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-31T00:46:41.881Z] "POST /llm/e2e-rt-sw-c292bd/anything/e2e-rt-sw-c292bd-client HTTP/1.1" 503 UH no_healthy_upstream - "-" 124 19 71 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "dd2248e4-0e7e-4332-be34-ca4e9233387c" "maas.apps.f7fad04c82cf1a4da903.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rt-sw-c292bd.llm.svc.cluster.local - 10.133.0.36:443 10.132.0.11:48386 - llm.external-model-e2e-rt-sw-c292bd-ef50224c2876.2 2026-10-31T00:47:08.284772Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-10-31T00:47:08.285031Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T00:47:08.285056Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T00:47:08.285067Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T00:47:08.285071Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T00:47:08.285250Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-10-31T00:47:08.210Z] "POST /llm/e2e-rt-sw-c292bd/anything/e2e-rt-sw-c292bd-client HTTP/1.1" 503 UH no_healthy_upstream - "-" 124 19 75 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "a9f73e38-7925-4c2e-9703-8a10784b3076" "maas.apps.f7fad04c82cf1a4da903.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rt-sw-c292bd.llm.svc.cluster.local - 10.133.0.36:443 10.132.0.11:60970 - llm.external-model-e2e-rt-sw-c292bd-ef50224c2876.2 2026-10-31T00:47:27.923881Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T00:47:47.671452Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T00:48:45.008458Z info ads ADS: "" 3 terminated 2026-10-31T00:48:47.313403Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T00:48:49.987394Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T00:49:12.842818Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T00:49:38.183974Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T00:50:12.004523Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T00:51:44.761049Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T00:52:03.119027Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T00:52:39.983601Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T00:52:39.983637Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T00:52:39.983653Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T00:52:39.983658Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-31T00:52:39.865Z] "DELETE /v1/api-keys/026c0269-bafc-41ab-baaa-2238bccc3df5 HTTP/1.1" 200 - via_upstream - "-" 0 321 117 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "87077219-c6fc-49f6-99c0-40395988b528" "maas.apps.f7fad04c82cf1a4da903.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.36:57458 10.133.0.36:443 10.132.0.11:46836 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T00:52:41.684771Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T00:52:41.684815Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-10-31T00:52:41.684836Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-10-31T00:52:41.684842Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-10-31T00:52:41.584Z] "DELETE /v1/api-keys/d61a26e2-9559-4863-9b51-fba5291dc41f HTTP/1.1" 200 - via_upstream - "-" 0 321 100 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "7e4d68ee-f1d1-415b-80bc-8d666a403d52" "maas.apps.f7fad04c82cf1a4da903.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.36:57458 10.133.0.36:443 10.132.0.11:46848 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T00:52:44.834070Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T00:52:44.834106Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T00:52:44.834121Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T00:52:44.834125Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T00:52:45.607565Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T00:52:45.607602Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T00:52:45.607616Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T00:52:45.607621Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-10-31T00:52:44.726Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 311 107 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "b399217b-96c9-49af-a61b-34f4f9193cb0" "maas.apps.f7fad04c82cf1a4da903.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.36:50602 10.133.0.36:443 10.132.0.11:46876 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T00:52:45.493Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 114 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "7f35756f-3946-402b-a320-caafe544ec9f" "maas.apps.f7fad04c82cf1a4da903.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.36:50602 10.133.0.36:443 10.132.0.11:43726 - odh-ai-gateway-infra.maas-api-route.3 2026-10-31T00:52:47.535066Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T00:52:47.535092Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T00:52:47.535096Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=25 2026-10-31T00:52:47.535102Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=25 [2026-10-31T00:52:47.499Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 77 35 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "5b2e4a8c-5c9c-46a9-9fb3-a3c60e43238b" "maas.apps.f7fad04c82cf1a4da903.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.36:443 10.132.0.11:43742 - v1-completions-path 2026-10-31T00:53:19.631275Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-10-31T00:53:54.646512Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-10-31T00:54:24.652Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 279 78 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "c24ca915-8425-4a99-8641-0197a3a2b03b" "maas.apps.f7fad04c82cf1a4da903.prod.konflux-ocp-ci.dev" "10.134.0.38:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.36:46276 10.133.0.36:443 10.132.0.11:43988 - v1-completions-path 2026-10-31T01:04:06.630788Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-10-31T01:04:05.697Z] "POST /maas-api/v1/tokens HTTP/1.1" 401 - - "-" 20 88 23 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "53a16969-fc5b-4d19-87d6-846f595ba23d" "maas.apps.f7fad04c82cf1a4da903.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.133.0.36:443 10.132.0.11:36792 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T01:04:06.561Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 136 391 77 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "1190c823-453d-4d06-80b5-16e1f66781e5" "maas.apps.f7fad04c82cf1a4da903.prod.konflux-ocp-ci.dev" "10.134.0.38:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.36:46276 10.133.0.36:443 10.132.0.11:36794 - v1-chat-completions-path [2026-10-31T01:08:01.613Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 113 321 69 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "d072a61a-182a-4666-bd28-4f79f412a451" "maas.apps.f7fad04c82cf1a4da903.prod.konflux-ocp-ci.dev" "10.134.0.38:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.36:46276 10.133.0.36:443 10.132.0.11:58816 - v1-chat-completions-path 2026-10-31T01:08:01.947260Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T01:08:01.947303Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T01:08:01.947323Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T01:08:01.947329Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T01:08:01.947513Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=24 2026-10-31T01:08:01.947573Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-31T01:08:01.947584Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-31T01:08:01.947588Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-10-31T01:08:01.947593Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-10-31T01:08:01.887Z] "POST /v1/chat/completions HTTP/1.1" 404 - via_upstream - "-" 126 129 60 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "0514666d-8744-43bc-b9f3-f1389274104c" "maas.apps.f7fad04c82cf1a4da903.prod.konflux-ocp-ci.dev" "10.134.0.38:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.36:46276 10.133.0.36:443 10.132.0.11:58832 - v1-chat-completions-path 2026-10-31T01:09:45.843579Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-10-31T01:13:33.435172Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T01:13:33.435207Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-10-31T01:13:33.435221Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-10-31T01:13:33.435226Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-10-31T01:13:33.314Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 69 312 120 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "ec493646-bed5-4dcb-a501-5da0dd72c051" "maas.apps.f7fad04c82cf1a4da903.prod.konflux-ocp-ci.dev" "10.134.0.29:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.36:37582 10.133.0.36:443 10.132.0.11:38796 - odh-ai-gateway-infra.maas-api-route.3 [2026-10-31T01:20:06.518Z] "GET /v1/tenants HTTP/1.1" 404 NR route_not_found - "-" 0 0 9 - "34.228.250.194,10.132.0.11" "python-requests/2.34.2" "f922a9b1-05b5-4bd8-9327-8eea1a7dcf0e" "maas.apps.f7fad04c82cf1a4da903.prod.konflux-ocp-ci.dev" "-" - - 10.133.0.36:443 10.132.0.11:53396 - -