2026-11-02T18:54:53.819594Z info FLAG: --concurrency="0" 2026-11-02T18:54:53.819612Z info FLAG: --domain="openshift-ingress.svc.cluster.local" 2026-11-02T18:54:53.819617Z info FLAG: --help="false" 2026-11-02T18:54:53.819619Z info FLAG: --log_as_json="false" 2026-11-02T18:54:53.819621Z info FLAG: --log_caller="" 2026-11-02T18:54:53.819623Z info FLAG: --log_output_level="default:info" 2026-11-02T18:54:53.819625Z info FLAG: --log_stacktrace_level="default:none" 2026-11-02T18:54:53.819629Z info FLAG: --log_target="[stdout]" 2026-11-02T18:54:53.819631Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2026-11-02T18:54:53.819634Z info FLAG: --outlierLogPath="" 2026-11-02T18:54:53.819636Z info FLAG: --profiling="true" 2026-11-02T18:54:53.819638Z info FLAG: --proxyComponentLogLevel="misc:error" 2026-11-02T18:54:53.819639Z info FLAG: --proxyLogLevel="warning" 2026-11-02T18:54:53.819641Z info FLAG: --serviceCluster="istio-proxy" 2026-11-02T18:54:53.819643Z info FLAG: --stsPort="0" 2026-11-02T18:54:53.819645Z info FLAG: --templateFile="" 2026-11-02T18:54:53.819647Z info FLAG: --tokenManagerPlugin="" 2026-11-02T18:54:53.819649Z info FLAG: --vklog="0" 2026-11-02T18:54:53.819652Z info Version 1.28.10-cebc47d567fd1333707199224c18129ecfe07c05-Clean 2026-11-02T18:54:53.819834Z info Proxy role ips=[10.134.1.72] type=router id=maas-default-gateway-openshift-default-9dccdc9b7-26plb.openshift-ingress domain=openshift-ingress.svc.cluster.local 2026-11-02T18:54:53.819890Z info Apply proxy config from env {"discoveryAddress":"istiod-openshift-gateway.openshift-ingress.svc:15012","proxyHeaders":{"server":{"disabled":true},"envoyDebugHeaders":{"disabled":true},"metadataExchangeHeaders":{"mode":"IN_MESH"}}} 2026-11-02T18:54:53.821255Z info cpu limit detected as 2, setting concurrency 2026-11-02T18:54:53.821459Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod-openshift-gateway.openshift-ingress.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyHeaders: envoyDebugHeaders: disabled: true metadataExchangeHeaders: mode: IN_MESH server: disabled: true serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2026-11-02T18:54:53.821471Z info JWT policy is third-party-jwt 2026-11-02T18:54:53.821473Z info using credential fetcher of JWT type in cluster.local trust domain 2026-11-02T18:54:53.823115Z info Starting default Istio SDS Server 2026-11-02T18:54:53.823143Z info CA Endpoint istiod-openshift-gateway.openshift-ingress.svc:15012, provider Citadel 2026-11-02T18:54:53.823160Z info Using CA istiod-openshift-gateway.openshift-ingress.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2026-11-02T18:54:53.823401Z info Opening status port 15020 2026-11-02T18:54:53.824358Z info xdsproxy Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes" 2026-11-02T18:54:53.826485Z info Pilot SAN: [istiod-openshift-gateway.openshift-ingress.svc] 2026-11-02T18:54:53.827712Z info Starting proxy agent 2026-11-02T18:54:53.827769Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2026-11-02T18:54:53.835930Z info sds Starting SDS grpc server 2026-11-02T18:54:53.835959Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2026-11-02T18:54:53.892841Z info xdsproxy connected to delta upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012 id=1 2026-11-02T18:54:53.903338Z info cache generated new workload certificate resourceName=default latency=79.102704ms ttl=23h59m59.096666588s 2026-11-02T18:54:53.903374Z info cache Root cert has changed, start rotating root cert 2026-11-02T18:54:53.903413Z info cache returned workload trust anchor from cache ttl=23h59m59.096587216s 2026-11-02T18:54:54.044540Z info ads ADS: new connection for node:1 2026-11-02T18:54:54.044610Z info cache returned workload certificate from cache ttl=23h59m58.955391437s 2026-11-02T18:54:54.045123Z info ads ADS: new connection for node:2 2026-11-02T18:54:54.045206Z info cache returned workload trust anchor from cache ttl=23h59m58.954796513s 2026-11-02T18:54:54.126339Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T18:55:02.089319Z info Readiness succeeded in 8.272316518s 2026-11-02T18:55:02.089697Z info Envoy proxy is ready 2026-11-02T18:55:29.123335Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T18:55:31.726671Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T18:55:31.726711Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T18:55:31.726725Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T18:55:31.726750Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-02T18:55:31.591Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 61 304 135 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "6a7e77a8-2ecd-4607-97da-bfa9cdf87794" "maas.apps.ee7d99a15883b5bfd98c.prod.konflux-ocp-ci.dev" "10.132.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.72:38960 10.134.1.72:443 10.132.0.19:57666 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T18:55:33.567Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 108118 108394 348 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "f4ab2d88-f338-4b4f-864d-811eee5f3256" "maas.apps.ee7d99a15883b5bfd98c.prod.konflux-ocp-ci.dev" "10.134.0.35:8000" outbound|8000||echo-simulator-kserve-workload-svc.llm.svc.cluster.local 10.134.1.72:42848 10.134.1.72:443 10.132.0.19:57684 - v1-chat-completions-path 2026-11-02T18:55:48.129177Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T18:55:52.742668Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T18:56:02.413243Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T18:56:05.445053Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T18:56:05.445110Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T18:56:05.445136Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T18:56:05.445145Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-02T18:56:05.263Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 317 182 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "5e507afb-6fe8-4ae2-93e3-e702c315d9a3" "maas.apps.ee7d99a15883b5bfd98c.prod.konflux-ocp-ci.dev" "10.132.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.72:38960 10.134.1.72:443 10.132.0.19:51178 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T18:56:15.613073Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T18:56:24.749406Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T18:56:41.470819Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T18:56:53.385095Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T18:56:56.245261Z info ads ADS: new connection for node:3 2026-11-02T18:56:56.245473Z info cache adding watcher for file certificate /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-02T18:56:56.245503Z info cache read certificate from file resource=file-root:/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt 2026-11-02T18:57:00.004316Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T18:57:39.776414Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T18:58:04.231165Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T18:58:04.258522Z info ads ADS: "" 3 terminated with stream closed 2026-11-02T18:58:09.054585Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T18:58:25.558607Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:00:58.629243Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:01:26.407642Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:01:52.741799Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:02:07.679745Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:02:14.223005Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:02:26.019675Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:02:30.822035Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:02:34.066285Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T19:02:34.066325Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T19:02:34.066342Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T19:02:34.066346Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-02T19:02:34.009Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 524 56 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "a6f50781-e169-443e-b924-2e14c5c49b7b" "maas.apps.ee7d99a15883b5bfd98c.prod.konflux-ocp-ci.dev" "10.132.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.72:52430 10.134.1.72:443 10.132.0.19:55308 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T19:02:34.893Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 153 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "89936897-f34f-40e5-be94-0eb8504d1934" "maas.apps.ee7d99a15883b5bfd98c.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.72:443 10.132.0.19:55310 - v1-completions-path 2026-11-02T19:02:49.141219Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-11-02T19:03:07.067Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 295 74 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "84ba4920-c2ec-4674-a461-e1b4f9f9f554" "maas.apps.ee7d99a15883b5bfd98c.prod.konflux-ocp-ci.dev" "10.132.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.1.72:41642 10.134.1.72:443 10.132.0.19:46534 - v1-completions-path 2026-11-02T19:03:37.627265Z info ads ADS: new connection for node:4 2026-11-02T19:03:37.632467Z info cache adding watcher for file certificate /etc/pki/tls/certs/ca-bundle.crt 2026-11-02T19:03:37.632521Z info cache read certificate from file resource=file-root:system 2026-11-02T19:03:41.588774Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:03:42.680425Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:03:50.376090Z warning envoy ext_proc external/envoy/source/extensions/filters/http/ext_proc/ext_proc.cc:1688 [Tags: "ConnectionId":"166","StreamId":"8288696610824105529"] Received gRPC error on stream: 14, message upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}} thread=26 [2026-11-02T19:03:50.312Z] "POST /models-as-a-service/e2e-rotation-external-model/anything/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 92 0 63 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "023054c5-7ae0-43b2-be93-502dd778dcbc" "maas.apps.ee7d99a15883b5bfd98c.prod.konflux-ocp-ci.dev" "-" outbound|443||provider-selection-required-e2e-rotation-external-model.models-as-a-service.svc.cluster.local - 10.134.1.72:443 10.132.0.19:38374 - models-as-a-service.external-model-e2e-rotation-external-model.2 2026-11-02T19:04:57.615332Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T19:04:57.615368Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T19:04:57.615375Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=25 2026-11-02T19:04:57.615384Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T19:04:57.615388Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 2026-11-02T19:04:57.615550Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1155 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Task failed: Some("2") thread=25 [2026-11-02T19:04:57.491Z] "POST /anything/v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 92 1713 123 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "964ab97f-655c-4e8d-91af-07dfa4b622b2" "httpbingo.org" "66.241.125.232:443" outbound|443||httpbingo.org 10.134.1.72:42512 10.134.1.72:443 10.132.0.19:60894 - models-as-a-service.external-model-e2e-rotation-external-model.0 2026-11-02T19:05:26.836577Z info ads ADS: "" 4 terminated 2026-11-02T19:05:26.880826Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:05:29.446209Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:05:39.965509Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:05:58.289369Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:06:02.696413Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-11-02T19:06:09.110Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 429 - - "-" 76 18 59 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "1ca8068d-2427-40a1-ace6-9b31d024c27d" "maas.apps.ee7d99a15883b5bfd98c.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.72:443 10.132.0.19:46250 - v1-completions-path 2026-11-02T19:06:26.510582Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:06:39.861286Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:06:46.326254Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T19:06:46.326292Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T19:06:46.326307Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=26 2026-11-02T19:06:46.326312Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=26 [2026-11-02T19:06:46.209Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 81 324 117 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "b4a864a0-01a1-492d-ae3a-748045e8cb16" "maas.apps.ee7d99a15883b5bfd98c.prod.konflux-ocp-ci.dev" "10.132.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.72:59576 10.134.1.72:443 10.132.0.19:55220 - odh-ai-gateway-infra.maas-api-route.3 [2026-11-02T19:06:47.260Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 80 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "8555e3c0-1ec8-4bac-a254-d753e66458de" "maas.apps.ee7d99a15883b5bfd98c.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.72:54276 10.134.1.72:443 10.132.0.19:55234 - v1-completions-path 2026-11-02T19:07:10.768464Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:07:21.507551Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:07:27.698477Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:07:40.320209Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Tracing span buffer full (100), dropping oldest spans to prevent memory growth thread=26 [2026-11-02T19:07:40.263Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 62 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "05441697-7b4a-410e-8b34-d0ebe74dbc6e" "maas.apps.ee7d99a15883b5bfd98c.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.72:42264 10.134.1.72:443 10.132.0.19:59484 - v1-completions-path 2026-11-02T19:07:56.294317Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:08:00.957027Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:08:14.046237Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:08:23.826970Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:08:33.067993Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:08:41.956782Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:08:49.561916Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-11-02T19:08:54.331Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 284 65 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "adb65448-8a1f-44b7-bab2-2acb0b153fd4" "maas.apps.ee7d99a15883b5bfd98c.prod.konflux-ocp-ci.dev" "10.132.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.1.72:41642 10.134.1.72:443 10.132.0.19:41056 - v1-completions-path 2026-11-02T19:09:09.637745Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:09:15.701503Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-11-02T19:09:29.860Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 36 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "1e58ba1d-9853-4aff-8cdd-86759a513c12" "maas.apps.ee7d99a15883b5bfd98c.prod.konflux-ocp-ci.dev" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.72:443 10.132.0.19:49090 - v1-completions-path 2026-11-02T19:09:33.984492Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:09:39.376372Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:09:41.624685Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T19:09:41.624717Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T19:09:41.624757Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /model thread=25 2026-11-02T19:09:41.624764Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log kuadrant-wasm-shim kuadrant_wasm_shim: Missing json property: /usage/total_tokens thread=25 [2026-11-02T19:09:41.512Z] "DELETE /v1/api-keys/cdc3d108-56a5-4358-9b04-012cdc3f4318 HTTP/1.1" 404 - via_upstream - "-" 0 29 111 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "ef595ae3-a3ec-4b4f-a1cf-de3060cea23f" "maas.apps.ee7d99a15883b5bfd98c.prod.konflux-ocp-ci.dev" "10.132.0.31:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.1.72:53292 10.134.1.72:443 10.132.0.19:40144 - odh-ai-gateway-infra.maas-api-route.3 2026-11-02T19:09:50.555615Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:10:17.702887Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:10:28.401645Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:10:49.479957Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:10:55.490646Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:13:47.284774Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:15:17.029661Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-11-02T19:15:19.754Z] "POST /llm/e2e-trlp-test-simulated/v1/completions HTTP/1.1" 429 - - "-" 73 18 98 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "bd1498d4-6ece-45d9-870d-345b56f951e8" "maas.apps.ee7d99a15883b5bfd98c.prod.konflux-ocp-ci.dev" "-" outbound|8000||e2e-trlp-test-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.1.72:443 10.132.0.19:52898 - v1-completions-path [2026-11-02T19:15:22.056Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 73 302 75 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "c90a0dfa-6d66-4c2f-bd5e-8b8571165ecb" "maas.apps.ee7d99a15883b5bfd98c.prod.konflux-ocp-ci.dev" "10.132.0.39:8000" outbound|8000||e2e-trlp-test-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.1.72:35984 10.134.1.72:443 10.132.0.19:52912 - v1-completions-path 2026-11-02T19:15:36.632046Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:15:41.657544Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:15:54.722915Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:16:11.157600Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:16:16.081397Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-11-02T19:16:19.289Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 292 73 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "8ee5dceb-8579-4d12-aa87-cae0416316c0" "maas.apps.ee7d99a15883b5bfd98c.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.72:42264 10.134.1.72:443 10.132.0.19:36460 - v1-completions-path 2026-11-02T19:16:35.890638Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:16:46.464274Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:16:52.677894Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-11-02T19:17:05.709Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 60 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "47513e67-4ce8-4444-8e35-a349e60285ad" "maas.apps.ee7d99a15883b5bfd98c.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.72:54276 10.134.1.72:443 10.132.0.19:59410 - v1-completions-path 2026-11-02T19:17:19.960499Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:17:26.162103Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:17:32.224418Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-11-02T19:17:44.281Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 64 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "b8b04ba5-20c2-4acf-a602-4ce07057b62d" "maas.apps.ee7d99a15883b5bfd98c.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.72:54276 10.134.1.72:443 10.132.0.19:53106 - v1-completions-path 2026-11-02T19:18:04.767478Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:18:15.620918Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:18:22.367473Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:18:28.386897Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-11-02T19:18:33.644Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 290 64 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "d520727b-5ed3-4739-8358-2847818b5fce" "maas.apps.ee7d99a15883b5bfd98c.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.72:42264 10.134.1.72:443 10.132.0.19:50222 - v1-completions-path 2026-11-02T19:18:37.627899Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 [2026-11-02T19:18:44.160Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 294 73 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "f7b2d95d-db4f-47d0-bb15-a5ad5aeb4309" "maas.apps.ee7d99a15883b5bfd98c.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.72:54276 10.134.1.72:443 10.132.0.19:54752 - v1-completions-path [2026-11-02T19:18:45.645Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 76 295 60 - "52.71.36.72,10.132.0.19" "python-requests/2.34.2" "e82716fa-3b1a-4a13-ad27-1173d920fa95" "maas.apps.ee7d99a15883b5bfd98c.prod.konflux-ocp-ci.dev" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.1.72:42264 10.134.1.72:443 10.132.0.19:60534 - v1-completions-path 2026-11-02T19:19:00.644312Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:19:11.348505Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:19:16.080553Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:19:34.504035Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:19:40.580796Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:20:04.499037Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:20:26.320278Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:20:46.430632Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:20:57.067658Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:21:07.861650Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:21:18.663541Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:21:29.355655Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:21:40.175933Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:21:50.913076Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7 2026-11-02T19:22:01.554454Z warning envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:317 Wasm remote code fetch is unstable and may cause a crash thread=7