2026-11-02T20:17:16.055812Z info FLAG: --concurrency="0" 2026-11-02T20:17:16.055829Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-02T20:17:16.055833Z info FLAG: --help="false" 2026-11-02T20:17:16.055835Z info FLAG: --log_as_json="false" 2026-11-02T20:17:16.055837Z info FLAG: --log_caller="" 2026-11-02T20:17:16.055839Z info FLAG: --log_output_level="default:info" 2026-11-02T20:17:16.055841Z info FLAG: --log_stacktrace_level="default:none" 2026-11-02T20:17:16.055845Z info FLAG: --log_target="[stdout]" 2026-11-02T20:17:16.055848Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-02T20:17:16.055850Z info FLAG: --outlierLogPath="" 2026-11-02T20:17:16.055852Z info FLAG: --profiling="true" 2026-11-02T20:17:16.055854Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-02T20:17:16.055856Z info FLAG: --proxyLogLevel="warning" 2026-11-02T20:17:16.055858Z info FLAG: --serviceCluster="istio-proxy" 2026-11-02T20:17:16.055860Z info FLAG: --stsPort="0" 2026-11-02T20:17:16.055861Z info FLAG: --templateFile="" 2026-11-02T20:17:16.055863Z info FLAG: --tokenManagerPlugin="" 2026-11-02T20:17:16.055866Z info FLAG: --vklog="0" 2026-11-02T20:17:16.055868Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-02T20:17:16.056001Z info Proxy role ips=[10.133.0.94] type=router id=maas-default-gateway-openshift-default-5db77fbd9f-ns7hq.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-02T20:17:16.056054Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-02T20:17:16.057086Z info cpu limit detected as 2, setting concurrency 2026-11-02T20:17:16.057316Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-02T20:17:16.057327Z info JWT policy is third-party-jwt 2026-11-02T20:17:16.057330Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-02T20:17:16.058979Z info Starting default Istio SDS Server 2026-11-02T20:17:16.058998Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-02T20:17:16.059021Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-02T20:17:16.059544Z info Opening status port 15020 2026-11-02T20:17:16.059971Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-02T20:17:16.060502Z info sds Starting SDS grpc server 2026-11-02T20:17:16.060517Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-02T20:17:16.061723Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-02T20:17:16.063907Z info Starting proxy agent 2026-11-02T20:17:16.063947Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-02T20:17:16.132461Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-02T20:17:16.228438Z info cache generated new workload certificate resourceName=default latency=167.811684ms ttl=23h59m59.771566782s 2026-11-02T20:17:16.228478Z info cache Root cert has changed, start rotating root cert 2026-11-02T20:17:16.228518Z info cache returned workload trust anchor from cache ttl=23h59m59.771483183s 2026-11-02T20:17:16.337227Z info ads ADS: new connection for node:1 2026-11-02T20:17:16.337296Z info cache returned workload certificate from cache ttl=23h59m59.662706395s 2026-11-02T20:17:16.337859Z info ads ADS: new connection for node:2 2026-11-02T20:17:16.337954Z info cache returned workload trust anchor from cache ttl=23h59m59.662047435s 2026-11-02T20:17:16.425069Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T20:17:24.591820Z info Readiness succeeded in 8.538608532s 2026-11-02T20:17:24.592638Z info Envoy proxy is ready 2026-11-02T20:17:25.434259Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T20:19:03.637362Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T20:20:47.020446Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T20:27:21.045Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 110 320 98 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "29b07748-dff3-4943-a9dc-797089b3d6a6" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.132.0.51:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.94:48670 10.133.0.94:443 10.132.0.22:47910 - v1-chat-completions-path 2026-11-02T20:30:07.363516Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=2 2026-11-02T20:30:19.156127Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T20:30:19.156169Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T20:30:19.156187Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T20:30:19.156194Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T20:30:19.137Z] "GET /health HTTP/1.1" 200 - via_upstream - "-" 0 91 18 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "22b0c0c5-c20b-4478-98f5-59aa6b8a3026" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.133.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.94:52966 10.133.0.94:443 10.132.0.22:58672 - odh-ai-gateway-infra.maas-api-route.4 [2026-11-02T20:30:20.034Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 136 322 86 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "6b15e1ea-2c5d-4a72-be1e-0ee91de08ae8" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.132.0.51:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.94:48670 10.133.0.94:443 10.132.0.22:58686 - v1-chat-completions-path 2026-11-02T20:31:14.509561Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T20:31:14.509594Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T20:31:14.509609Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T20:31:14.509613Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-02T20:31:14.397Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 72 314 112 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "0ed67a40-f4cb-4efd-95b7-ad11f3fe115c" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.133.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.94:52966 10.133.0.94:443 10.132.0.22:37790 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T20:31:14.697Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 113 318 73 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "5f2db295-b75e-4b51-83ae-52cfec7771b8" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.132.0.51:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.94:51626 10.133.0.94:443 10.132.0.22:37796 - v1-chat-completions-path [2026-11-02T20:31:14.976Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 113 326 60 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "865dd9ea-022f-4507-8045-5891b47e64be" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.132.0.51:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.94:51626 10.133.0.94:443 10.132.0.22:37800 - v1-chat-completions-path [2026-11-02T20:31:27.145Z] "POST /maas-api/v1/tokens HTTP/1.1" 401 - - "-" 20 88 31 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "c76b459e-1c5e-4963-922e-63b2c021c401" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "-" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local - 10.133.0.94:443 10.132.0.22:59164 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T20:45:19.215Z] "GET /v1/tenants HTTP/1.1" 404 NR route_not_found - "-" 0 0 9 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "66099321-316c-4741-800d-b495012edf3e" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "-" - - 10.133.0.94:443 10.132.0.22:44878 - - 2026-11-02T20:51:59.278557Z info ads ADS: new connection for node:3 2026-11-02T20:51:59.284244Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-02T20:51:59.284303Z info cache read certificate from file resource=file-root:system 2026-11-02T20:52:02.346695Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T20:52:13.204504Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T20:54:00.386310Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T20:54:42.681831Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T20:56:01.532965Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T20:56:37.098Z] "POST /llm/e2e-dft-c700ba/anything/e2e-dft-c700ba-4b93a4 HTTP/1.1" 503 - - "-" 109 0 120 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "22b69f0a-911d-4e6c-82f3-e19337c37449" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-c700ba.llm.svc.cluster.local - 10.133.0.94:443 10.132.0.22:53080 - llm.external-model-e2e-dft-c700ba-d669b1a95487.2 2026-11-02T20:57:35.827790Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=24 [2026-11-02T20:57:35.746Z] "POST /llm/e2e-dft-c700ba/anything/e2e-dft-c700ba-4b93a4 HTTP/1.1" 503 - - "-" 109 0 82 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "e9cbcc08-956d-436f-b3c6-8a6ce51e1cfa" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-c700ba.llm.svc.cluster.local - 10.133.0.94:443 10.132.0.22:39484 - llm.external-model-e2e-dft-c700ba-d669b1a95487.2 2026-11-02T21:00:22.627071Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:00:34.482700Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=3 [2026-11-02T21:00:52.597Z] "POST /llm/e2e-dft-c700ba/anything/e2e-dft-c700ba-4b93a4 HTTP/1.1" 403 - - "-" 109 77 49 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "d0912e5b-9006-4d56-b17c-dae344e3044c" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-dft-c700ba.llm.svc.cluster.local - 10.133.0.94:443 10.132.0.22:54876 - llm.external-model-e2e-dft-c700ba-d669b1a95487.2 2026-11-02T21:01:05.823093Z info ads ADS: "" 3 terminated with stream closed 2026-11-02T21:01:06.792239Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:01:08.851954Z info ads ADS: new connection for node:4 2026-11-02T21:01:08.868352Z info cache read certificate from file resource=file-root:system 2026-11-02T21:01:12.703512Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:01:14.523825Z info ads ADS: "" 4 terminated with stream closed 2026-11-02T21:01:16.297262Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:01:21.218361Z info ads ADS: new connection for node:5 2026-11-02T21:01:21.234262Z info cache read certificate from file resource=file-root:system 2026-11-02T21:01:23.383033Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:01:27.233009Z info ads ADS: "" 5 terminated 2026-11-02T21:01:28.019481Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:01:45.228292Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:02:43.357475Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:04:52.044800Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:19:44.606410Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-02T21:19:44.619983Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T21:19:44.620014Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T21:19:44.620029Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T21:19:44.620034Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T21:19:44.446Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 30 315 173 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "8fbfdeac-b464-45e4-994c-aea6dab00f75" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.133.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.94:50248 10.133.0.94:443 10.132.0.22:50672 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T21:19:49.458050Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:19:49.458085Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:19:49.458100Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:19:49.458105Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-02T21:19:49.346Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 313 111 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "d4f67053-3714-49b3-968b-b31dd7908778" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.133.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.94:50254 10.133.0.94:443 10.132.0.22:52292 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T21:19:51.994748Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:20:16.141880Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:20:25.547146Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T21:20:25.547178Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T21:20:25.547191Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T21:20:25.547196Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 [2026-11-02T21:20:25.434Z] "POST /v1/api-keys HTTP/1.1" 400 - via_upstream - "-" 80 94 113 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "70242d59-8742-41f3-aea1-7089db9973f0" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.133.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.94:50248 10.133.0.94:443 10.132.0.22:58356 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T21:20:52.691762Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:20:55.241337Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:20:55.241373Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:20:55.241389Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T21:20:55.241394Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-02T21:20:55.120Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 61 303 121 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "56a7b013-8fc2-4847-8b5d-53abe6d07510" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.133.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.94:53508 10.133.0.94:443 10.132.0.22:43064 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T21:20:56.859Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 5518 5791 76 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "41e2e1f2-3c35-464c-9960-2241826e291a" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.133.0.39:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.133.0.94:40696 10.133.0.94:443 10.132.0.22:57004 - v1-chat-completions-path [2026-11-02T21:20:57.759Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 144 409 63 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "da891eca-e130-4890-9d98-900f920d38ff" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.133.0.39:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.133.0.94:40712 10.133.0.94:443 10.132.0.22:57008 - v1-chat-completions-path [2026-11-02T21:20:58.127Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 140 405 58 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "e63dfe8a-6614-4fc0-891e-056d26b6b692" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.133.0.39:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.133.0.94:40712 10.133.0.94:443 10.132.0.22:57022 - v1-chat-completions-path 2026-11-02T21:21:10.476512Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-02T21:21:10.476538Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-02T21:21:10.476541Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:372 thread=24 2026-11-02T21:21:10.476545Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'model' missing from lines 1-1 key: model thread=24 [2026-11-02T21:21:10.318Z] "POST /llm/e2e-embedding-simulated/v1/embeddings HTTP/1.1" 403 - - "-" 61 103 157 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "0bab5450-9707-4c57-b8b2-4763fa7d1d1c" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.94:443 10.132.0.22:52542 - v1-catch-all-path 2026-11-02T21:21:11.624639Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:21:16.201886Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:21:18.533573Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-11-02T21:21:18.452Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 81 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "cc921d26-b353-4225-a98b-fc2f3460c0b5" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.132.0.49:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.94:57118 10.133.0.94:443 10.132.0.22:51934 - v1-catch-all-path 2026-11-02T21:21:25.091242Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:21:27.621785Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-11-02T21:21:27.559Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 62 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "1582b2ea-4524-4680-84d5-47544f0c6a7b" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.132.0.49:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.94:57118 10.133.0.94:443 10.132.0.22:44514 - v1-catch-all-path 2026-11-02T21:21:29.872907Z warning envoy filter external/envoy/source/extensions/filters/http/json_to_metadata/filter.cc:385 key 'completion_tokens' missing from lines 1-1 key: completion_tokens thread=24 [2026-11-02T21:21:29.813Z] "POST /v1/embeddings HTTP/1.1" 200 - via_upstream - "-" 61 4185 59 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "fa1e7dda-69b6-4524-b4c0-86f57001a218" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.132.0.49:8000" outbound|8000||e2e-embedding-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.94:57118 10.133.0.94:443 10.132.0.22:44528 - v1-catch-all-path 2026-11-02T21:21:37.755486Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:21:46.307105Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:21:49.208760Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T21:21:49.208798Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T21:21:49.208813Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T21:21:49.208819Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T21:21:49.093Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 73 315 115 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "ac023543-48ee-4d03-abe0-1609f7441fba" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.133.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.94:51164 10.133.0.94:443 10.132.0.22:39272 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T21:22:03.095916Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:22:15.283832Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:22:17.746034Z info ads ADS: new connection for node:6 2026-11-02T21:22:17.746190Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-02T21:22:17.746217Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-02T21:22:22.300957Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:23:05.630102Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T21:23:11.729Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 345 89 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "ee3d49af-767c-4e01-afe0-87ee1f22ab4b" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.133.0.212:8000" outbound|54321||e2e-prio-pool-006730b5-inference-pool-ip-8d0ecc4a.llm.svc.cluster.local 10.133.0.94:57968 10.133.0.94:443 10.132.0.22:47122 - v1-model-routing [2026-11-02T21:23:12.015Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 131 332 67 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "a785743d-485f-4aaa-ad0e-3e074203b951" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.133.0.212:8000" outbound|54321||e2e-prio-pool-006730b5-inference-pool-ip-8d0ecc4a.llm.svc.cluster.local 10.133.0.94:57968 10.133.0.94:443 10.132.0.22:47138 - v1-model-routing [2026-11-02T21:23:27.273Z] "POST /v1/chat/completions HTTP/1.1" 403 - - "-" 131 0 36 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "a9d51f4d-e2a6-43bc-a8c5-a8bfd8198ffd" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "-" outbound|54321||e2e-prio-pool-006730b5-inference-pool-ip-8d0ecc4a.llm.svc.cluster.local - 10.133.0.94:443 10.132.0.22:38990 - v1-model-routing 2026-11-02T21:23:29.239337Z info ads ADS: "" 6 terminated 2026-11-02T21:23:29.537265Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:23:34.169041Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:23:49.985102Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:26:04.793582Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T21:26:04.793615Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T21:26:04.793627Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T21:26:04.793633Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T21:26:04.734Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 58 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "06c3e7b0-aec2-42f0-a14b-acb7de1ce552" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.133.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.94:35250 10.133.0.94:443 10.132.0.22:38002 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T21:26:23.301896Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:26:50.482423Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:27:21.512255Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:27:36.672829Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:27:42.864547Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:27:54.355931Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:27:57.337104Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:28:01.701204Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:28:01.701250Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:28:01.701273Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:28:01.701280Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-02T21:28:01.073Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 68 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "b814e689-602e-4d18-82a9-3b2374c1307e" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.94:443 10.132.0.22:38704 - v1-completions-path [2026-11-02T21:28:01.647Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 524 54 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "4214fd6b-5d05-4b18-ac72-b9a67f5d3469" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.133.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.94:47206 10.133.0.94:443 10.132.0.22:38722 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T21:28:15.318540Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:29:03.086294Z info ads ADS: new connection for node:7 2026-11-02T21:29:03.090970Z info cache read certificate from file resource=file-root:system 2026-11-02T21:29:06.700426Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:29:08.028043Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:30:23.524476Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:30:23.524512Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:30:23.524528Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 2026-11-02T21:30:23.524533Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 [2026-11-02T21:30:23.404Z] "DELETE /v1/api-keys/3d4a2114-58a3-42ad-a7ef-0e64c0e6eee5 HTTP/1.1" 200 - via_upstream - "-" 0 359 120 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "51b82125-f26a-4684-b82f-150ce683fbee" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.133.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.94:41822 10.133.0.94:443 10.132.0.22:33976 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T21:30:36.664070Z info ads ADS: "" 7 terminated with stream closed 2026-11-02T21:30:36.737475Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:30:39.252309Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:30:49.501175Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:30:53.027067Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=4 [2026-11-02T21:30:52.643Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 293 77 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "3a944998-7953-4d62-824f-e6065441bcdf" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.132.0.50:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.94:37486 10.133.0.94:443 10.132.0.22:34590 - v1-completions-path 2026-11-02T21:31:07.738929Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:31:12.182319Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T21:31:15.250Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 289 64 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "7433c510-e96b-4360-a6fd-ce69bb34e6d4" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.132.0.50:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.94:46112 10.133.0.94:443 10.132.0.22:42900 - v1-completions-path [2026-11-02T21:31:17.417Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 288 60 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "ab28659d-6554-4590-b00f-8f6b9b422f5b" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.132.0.50:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.94:46112 10.133.0.94:443 10.132.0.22:33534 - v1-completions-path 2026-11-02T21:31:32.237579Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:31:50.684407Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T21:31:58.452Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 61 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "d2030b57-c670-419c-9778-7e1fa487c0b7" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.132.0.50:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.94:37486 10.133.0.94:443 10.132.0.22:50560 - v1-completions-path [2026-11-02T21:31:58.716Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 60 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "e4c6c7c2-3bf4-49d5-964f-56a7c6bbcce8" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.132.0.50:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.94:46112 10.133.0.94:443 10.132.0.22:50572 - v1-completions-path [2026-11-02T21:32:01.509Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 291 58 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "40784c27-ee5b-499d-bcfa-a2612f62c106" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.132.0.50:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.94:37486 10.133.0.94:443 10.132.0.22:50574 - v1-completions-path 2026-11-02T21:32:19.192871Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:32:29.833874Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:32:36.037266Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T21:32:46.932Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 297 62 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "efdeb091-5561-40f1-a6a7-c1285abb6a54" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.132.0.50:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.94:46112 10.133.0.94:443 10.132.0.22:55958 - v1-completions-path [2026-11-02T21:32:47.472Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 67 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "8cc022fe-6f0e-41ec-9598-e2f57bfe0af7" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.132.0.50:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.94:46112 10.133.0.94:443 10.132.0.22:55968 - v1-completions-path [2026-11-02T21:32:50.099Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 57 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "71b81db8-edeb-4e4d-9958-60c8ec5b3f17" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.132.0.50:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.94:37486 10.133.0.94:443 10.132.0.22:55978 - v1-completions-path 2026-11-02T21:33:04.416030Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:33:08.709552Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T21:33:09.324Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 74 300 112 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "ca2537d0-db2b-4958-bf38-345253a8e113" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.132.0.52:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.133.0.94:37142 10.133.0.94:443 10.132.0.22:58584 - v1-completions-path 2026-11-02T21:33:21.509155Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:33:30.234974Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:33:39.370023Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:33:45.382538Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:33:59.293941Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:34:14.351717Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:34:20.051090Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:34:38.003863Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:34:43.272342Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T21:34:42.662Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 66 18 62 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "f4869bb5-bc54-401e-b613-f294e43fe17d" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.94:443 10.132.0.22:45032 - v1-completions-path 2026-11-02T21:34:54.049155Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:35:18.042700Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:35:28.761836Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:35:49.092640Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:35:56.508092Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:38:22.157675Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:39:51.805359Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:40:08.990627Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:40:13.758599Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:40:14.251950Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:40:14.251999Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:40:14.252022Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:40:14.252030Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-02T21:40:14.112Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 65 308 139 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "7a62c027-d911-458a-8f5b-c034efeb8d89" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.133.0.33:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.94:47504 10.133.0.94:443 10.132.0.22:56720 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T21:40:26.440652Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:40:42.640907Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:40:48.982982Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T21:40:53.771Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 61 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "fb5e969c-f0dc-4064-a8f6-7a9a1b49d06f" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.132.0.50:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.94:37486 10.133.0.94:443 10.132.0.22:52966 - v1-completions-path 2026-11-02T21:41:08.577816Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:41:19.273555Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:41:25.403104Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T21:41:38.310Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 56 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "9c25c256-4065-4097-b243-29dd2636d292" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.94:443 10.132.0.22:35386 - v1-completions-path 2026-11-02T21:41:52.663827Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:41:58.826171Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:42:04.592530Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:42:21.575596Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:42:21.575631Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:42:21.575645Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=24 2026-11-02T21:42:21.575650Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=24 [2026-11-02T21:42:21.453Z] "DELETE /v1/api-keys/ee86108d-ca84-4fa1-90ab-e73e6e2a3f34 HTTP/1.1" 200 - via_upstream - "-" 0 365 121 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "8897245c-2dc2-479f-b1f0-566e3a41128a" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "10.134.0.198:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.133.0.94:58858 10.133.0.94:443 10.132.0.22:59696 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T21:42:35.078488Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:42:45.883722Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:42:52.059045Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:42:57.910005Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:43:07.167185Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:43:30.331836Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:43:41.078682Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:43:45.649090Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:44:03.884262Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:44:10.811305Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:44:20.082349Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:44:41.665673Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:45:01.706556Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:45:14.299280Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:45:33.874509Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:45:44.693714Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:45:55.361773Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:46:06.098755Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:46:16.897449Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 2026-11-02T21:46:27.434095Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=8 [2026-11-02T21:46:37.132Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 39 - "34.228.250.194,10.132.0.22" "python-requests/2.34.2" "5906c59c-b5f1-4ec1-96c3-b3ce41e6c020" "maas.apps.6fa4745d83b81ed2e5ae.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.94:443 10.132.0.22:44766 - v1-completions-path