2026-07-15T21:35:56.052056Z info ads XDS: Incremental Pushing ConnectedEndpoints:4 Version:2026-07-15T21:35:51Z/429 2026-07-15T21:35:58.448512Z info ads Push debounce stable[807] 3 for config EnvoyFilter/openshift-ingress/payload-processing and 2 more configs: 100.086212ms since last change, 176.704251ms since last push, full=true 2026-07-15T21:35:58.449895Z info ads XDS: Pushing Services:108 ConnectedEndpoints:4 Version:2026-07-15T21:35:58Z/430 2026-07-15T21:35:58.451212Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:141 removed:0 size:162.1kB cached:135/138 2026-07-15T21:35:58.451491Z info delta CDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:141 removed:0 size:162.1kB cached:135/138 2026-07-15T21:35:58.451592Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:139 removed:0 size:161.6kB cached:135/138 2026-07-15T21:35:58.451845Z info delta CDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:141 removed:0 size:162.1kB cached:135/138 2026-07-15T21:35:58.451989Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:138 removed:0 size:33.4kB empty:0 cached:136/138 2026-07-15T21:35:58.452291Z info delta EDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:138 removed:0 size:33.4kB empty:0 cached:138/138 2026-07-15T21:35:58.452356Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:138 removed:0 size:33.4kB empty:0 cached:138/138 2026-07-15T21:35:58.452580Z info delta LDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:35:58.452630Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:35:58.452663Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:35:58.452745Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-07-15T21:35:58.452885Z info delta EDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:138 removed:0 size:33.4kB empty:0 cached:138/138 2026-07-15T21:35:58.452979Z info delta RDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:11.9kB cached:0/0 2026-07-15T21:35:58.453234Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.6kB 2026-07-15T21:35:58.453555Z info delta RDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:3.8kB cached:0/0 2026-07-15T21:35:58.454166Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:51.7kB cached:0/0 2026-07-15T21:35:58.454196Z info delta ECDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:8.4kB 2026-07-15T21:35:58.455471Z info delta ECDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:30.5kB 2026-07-15T21:35:58.465424Z info delta ECDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:135.6kB 2026-07-15T21:35:59.618495Z info ads Push Status: { "pilot_duplicate_envoy_clusters": { "kuadrant-auth-service": { "proxy": "e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress", "message": "Duplicate cluster kuadrant-auth-service found while pushing CDS" } } } 2026-07-15T21:36:00.194340Z info ads Push debounce stable[808] 1 for config Secret/ai-tenant-e2e-shared-a-677131/rate-test-model-677131-kserve-self-signed-certs: 100.163594ms since last change, 100.163521ms since last push, full=false 2026-07-15T21:36:00.194415Z info ads XDS: Incremental Pushing ConnectedEndpoints:4 Version:2026-07-15T21:35:58Z/430 2026-07-15T21:36:06.542144Z info model Incremental push, service rate-test-model-266bf8-kserve-workload-svc.ai-tenant-e2e-shared-b-266bf8.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2026-07-15T21:36:06.749616Z info ads Push debounce stable[809] 7 for config VirtualService/odh-ai-gateway-infra/openshift-ingress~e2e-shared-b-266bf8-istio-autogenerated-k8s-gateway-https~* and 4 more configs: 100.244286ms since last change, 254.894094ms since last push, full=true 2026-07-15T21:36:06.750299Z info ads XDS: Pushing Services:109 ConnectedEndpoints:4 Version:2026-07-15T21:36:06Z/431 2026-07-15T21:36:06.750976Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:4 removed:0 size:1.8kB cached:0/1 2026-07-15T21:36:06.750995Z info delta CDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:4 removed:0 size:1.8kB cached:0/1 2026-07-15T21:36:06.751046Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:2 removed:0 size:1.4kB cached:0/1 2026-07-15T21:36:06.751137Z info delta CDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:4 removed:0 size:1.8kB cached:0/1 2026-07-15T21:36:06.751902Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:138 removed:0 size:33.4kB empty:0 cached:138/138 2026-07-15T21:36:06.751920Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:138 removed:0 size:33.4kB empty:0 cached:138/138 2026-07-15T21:36:06.751924Z info delta EDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:138 removed:0 size:33.4kB empty:0 cached:138/138 2026-07-15T21:36:06.751966Z info delta EDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:138 removed:0 size:33.4kB empty:0 cached:138/138 2026-07-15T21:36:06.752181Z info delta LDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:36:06.752181Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:36:06.752475Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:36:06.752542Z info delta RDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:11.9kB cached:0/0 2026-07-15T21:36:06.752602Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-07-15T21:36:06.752636Z info delta SDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:4.5kB cached:1/1 2026-07-15T21:36:06.752453Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.6kB 2026-07-15T21:36:06.752791Z info delta SDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:4.6kB cached:1/1 2026-07-15T21:36:06.753155Z info delta RDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:12.8kB cached:0/0 2026-07-15T21:36:06.753193Z info delta SDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.5kB cached:1/1 2026-07-15T21:36:06.753580Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:51.7kB cached:0/0 2026-07-15T21:36:06.753631Z info delta SDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.5kB cached:1/1 2026-07-15T21:36:06.755452Z info delta EDS: PUSH request for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:107B empty:1 cached:0/1 filtered:138 2026-07-15T21:36:06.755603Z info delta EDS: PUSH request for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:107B empty:0 cached:1/1 filtered:138 2026-07-15T21:36:06.755706Z info delta EDS: PUSH request for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:107B empty:0 cached:1/1 filtered:138 2026-07-15T21:36:06.756223Z info delta EDS: PUSH request for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:107B empty:0 cached:1/1 filtered:138 2026-07-15T21:36:07.967106Z info model Full push, new service ai-tenant-e2e-shared-b-266bf8/rate-test-model-266bf8-kserve-workload-svc.ai-tenant-e2e-shared-b-266bf8.svc.cluster.local 2026-07-15T21:36:08.067402Z info ads Push debounce stable[810] 1 for config ServiceEntry/ai-tenant-e2e-shared-b-266bf8/rate-test-model-266bf8-kserve-workload-svc.ai-tenant-e2e-shared-b-266bf8.svc.cluster.local: 100.166067ms since last change, 100.165917ms since last push, full=true 2026-07-15T21:36:08.068017Z info ads XDS: Pushing Services:109 ConnectedEndpoints:4 Version:2026-07-15T21:36:08Z/432 2026-07-15T21:36:08.068583Z info delta CDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:4 removed:0 size:1.8kB cached:0/1 2026-07-15T21:36:08.068652Z info delta EDS: PUSH INC for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:107B empty:1 cached:0/1 2026-07-15T21:36:08.068749Z info delta CDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:4 removed:0 size:1.8kB cached:0/1 2026-07-15T21:36:08.068799Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:4 removed:0 size:1.8kB cached:0/1 2026-07-15T21:36:08.068843Z info delta EDS: PUSH INC for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:107B empty:0 cached:1/1 2026-07-15T21:36:08.068858Z info delta EDS: PUSH INC for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:107B empty:0 cached:1/1 2026-07-15T21:36:08.068583Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:2 removed:0 size:1.4kB cached:0/1 2026-07-15T21:36:08.069010Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.6kB 2026-07-15T21:36:08.069065Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:36:08.069067Z info delta LDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:36:08.069432Z info delta RDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:12.8kB cached:0/0 2026-07-15T21:36:08.069071Z info delta EDS: PUSH INC for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:107B empty:0 cached:1/1 2026-07-15T21:36:08.069520Z info delta RDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:11.9kB cached:0/0 2026-07-15T21:36:08.069724Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:36:08.069967Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-07-15T21:36:08.070349Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:51.7kB cached:0/0 2026-07-15T21:36:09.618661Z info ads Push Status: { "pilot_duplicate_envoy_clusters": { "kuadrant-auth-service": { "proxy": "maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress", "message": "Duplicate cluster kuadrant-auth-service found while pushing CDS" } }, "pilot_eds_no_instances": { "outbound|8000||rate-test-model-266bf8-kserve-workload-svc.ai-tenant-e2e-shared-b-266bf8.svc.cluster.local": {} } } 2026-07-15T21:36:11.712223Z info ads Push debounce stable[811] 1 for config WasmPlugin/openshift-ingress/kuadrant-e2e-shared-b-266bf8: 100.223657ms since last change, 100.223595ms since last push, full=true 2026-07-15T21:36:11.725380Z info ads XDS: Pushing Services:109 ConnectedEndpoints:4 Version:2026-07-15T21:36:11Z/433 2026-07-15T21:36:11.725713Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:36:11.725750Z info delta LDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:36:11.725824Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:36:11.725911Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.6kB 2026-07-15T21:36:11.728004Z info delta ECDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:30.5kB 2026-07-15T21:36:11.728069Z info delta ECDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:30.5kB 2026-07-15T21:36:11.733351Z info delta ECDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:135.6kB 2026-07-15T21:36:17.121609Z info ads Push debounce stable[812] 1 for config ServiceEntry/ai-tenant-e2e-shared-b-266bf8/rate-test-model-266bf8-kserve-workload-svc.ai-tenant-e2e-shared-b-266bf8.svc.cluster.local: 100.561119ms since last change, 100.561055ms since last push, full=false 2026-07-15T21:36:17.121677Z info ads XDS: Incremental Pushing ConnectedEndpoints:4 Version:2026-07-15T21:36:11Z/433 2026-07-15T21:36:25.492817Z info ads Push debounce stable[813] 1 for config Secret/ai-tenant-e2e-shared-b-266bf8/rate-test-model-266bf8-kserve-self-signed-certs: 100.120428ms since last change, 100.120337ms since last push, full=false 2026-07-15T21:36:25.492881Z info ads XDS: Incremental Pushing ConnectedEndpoints:4 Version:2026-07-15T21:36:11Z/433 2026-07-15T21:36:36.008665Z info ads Push debounce stable[814] 1 for config WasmPlugin/openshift-ingress/kuadrant-e2e-shared-b-266bf8: 100.956372ms since last change, 100.956319ms since last push, full=true 2026-07-15T21:36:36.021896Z info ads XDS: Pushing Services:109 ConnectedEndpoints:4 Version:2026-07-15T21:36:36Z/434 2026-07-15T21:36:36.022200Z info delta LDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:36:36.022215Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:36:36.022278Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:36:36.022201Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.6kB 2026-07-15T21:36:36.024981Z info delta ECDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:30.5kB 2026-07-15T21:36:36.026682Z info delta ECDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:31.8kB 2026-07-15T21:36:36.035043Z info delta ECDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:135.6kB 2026-07-15T21:36:36.109949Z info ads Push debounce stable[815] 1 for config WasmPlugin/openshift-ingress/kuadrant-e2e-shared-a-677131: 100.575032ms since last change, 100.574955ms since last push, full=true 2026-07-15T21:36:36.123251Z info ads XDS: Pushing Services:109 ConnectedEndpoints:4 Version:2026-07-15T21:36:36Z/435 2026-07-15T21:36:36.123610Z info delta LDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:36:36.123635Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.6kB 2026-07-15T21:36:36.123633Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:36:36.123634Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:36:36.125413Z info delta ECDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:31.8kB 2026-07-15T21:36:36.125742Z info delta ECDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:31.8kB 2026-07-15T21:36:36.131325Z info delta ECDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:135.6kB 2026-07-15T21:36:52.905218Z info ads Push debounce stable[816] 2 for config WasmPlugin/openshift-ingress/kuadrant-e2e-shared-a-677131 and 1 more configs: 100.330105ms since last change, 189.749634ms since last push, full=true 2026-07-15T21:36:52.919274Z info ads XDS: Pushing Services:109 ConnectedEndpoints:4 Version:2026-07-15T21:36:52Z/436 2026-07-15T21:36:52.919594Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:36:52.919623Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:36:52.919663Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.6kB 2026-07-15T21:36:52.919796Z info delta LDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:36:52.921348Z info delta ECDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:31.8kB 2026-07-15T21:36:52.921866Z info delta ECDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:31.8kB 2026-07-15T21:36:52.926838Z info delta ECDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:135.6kB 2026-07-15T21:36:58.489446Z info model Incremental push, service rate-test-model-677131-kserve-workload-svc.ai-tenant-e2e-shared-a-677131.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2026-07-15T21:36:58.492611Z error status Encountered unexpected error updating status for &{{gateway.networking.k8s.io/v1beta1/HTTPRoute 07fa2b0d-5fd8-4db8-8276-9bc280507b17 rate-test-model-677131-kserve-route ai-tenant-e2e-shared-a-677131 cluster.local map[app.kubernetes.io/component:llminferenceservice-router app.kubernetes.io/name:rate-test-model-677131 app.kubernetes.io/part-of:llminferenceservice] map[] 62916 2026-07-15 21:35:45 +0000 UTC [{serving.kserve.io/v1alpha2 LLMInferenceService rate-test-model-677131 c6ac1e0e-c9ab-432a-9c8e-23711a945014 0xc0050d3397 0xc0050d3396}] 1} 0xc002eca408 0xc00367d428 map[]}, will try again later: httproutes.gateway.networking.k8s.io "rate-test-model-677131-kserve-route" not found 2026-07-15T21:36:58.590231Z info ads Push debounce stable[817] 6 for config Secret/ai-tenant-e2e-shared-a-677131/kserve-metrics-reader-sa-dockercfg-jcw5b and 4 more configs: 100.737915ms since last change, 144.585197ms since last push, full=true 2026-07-15T21:36:58.590927Z info ads XDS: Pushing Services:108 ConnectedEndpoints:4 Version:2026-07-15T21:36:58Z/437 2026-07-15T21:36:58.591462Z info delta CDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:3 removed:1 size:507B 2026-07-15T21:36:58.591549Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:1 size:44B 2026-07-15T21:36:58.591567Z info delta CDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:3 removed:1 size:507B 2026-07-15T21:36:58.591557Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:3 removed:1 size:507B 2026-07-15T21:36:58.592427Z info delta EDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:139 removed:0 size:33.5kB empty:1 cached:138/139 2026-07-15T21:36:58.592512Z info delta EDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:139 removed:0 size:33.5kB empty:1 cached:138/139 2026-07-15T21:36:58.592613Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:139 removed:0 size:33.5kB empty:1 cached:138/139 2026-07-15T21:36:58.592684Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:139 removed:0 size:33.5kB empty:1 cached:138/139 2026-07-15T21:36:58.592734Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.6kB 2026-07-15T21:36:58.592887Z info delta LDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:36:58.593090Z info delta RDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:2.8kB cached:0/0 2026-07-15T21:36:58.593186Z info delta SDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:4.5kB cached:1/1 2026-07-15T21:36:58.593108Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:36:58.593303Z info delta RDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:12.8kB cached:0/0 2026-07-15T21:36:58.593371Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-07-15T21:36:58.593455Z info delta SDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:4.6kB cached:1/1 2026-07-15T21:36:58.592926Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:36:58.593392Z info delta SDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.5kB cached:1/1 2026-07-15T21:36:58.594572Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:51.7kB cached:0/0 2026-07-15T21:36:58.594667Z info delta SDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.5kB cached:1/1 2026-07-15T21:36:59.203462Z info ads Push debounce stable[818] 1 for config Secret/ai-tenant-e2e-shared-b-266bf8/kserve-metrics-reader-sa-dockercfg-d85t9: 100.178986ms since last change, 100.178893ms since last push, full=false 2026-07-15T21:36:59.203535Z info ads XDS: Incremental Pushing ConnectedEndpoints:4 Version:2026-07-15T21:36:58Z/437 2026-07-15T21:36:59.362426Z error status Encountered unexpected error updating status for &{{gateway.networking.k8s.io/v1beta1/HTTPRoute 74b42ad5-68bd-46b9-8c0b-93859734dc13 rate-test-model-266bf8-kserve-route ai-tenant-e2e-shared-b-266bf8 cluster.local map[app.kubernetes.io/component:llminferenceservice-router app.kubernetes.io/name:rate-test-model-266bf8 app.kubernetes.io/part-of:llminferenceservice] map[] 62913 2026-07-15 21:36:06 +0000 UTC [{serving.kserve.io/v1alpha2 LLMInferenceService rate-test-model-266bf8 97636804-b4a8-4266-8801-41f40be48512 0xc005d8de3d 0xc005d8de3c}] 1} 0xc00314e408 0xc003693048 map[]}, will try again later: httproutes.gateway.networking.k8s.io "rate-test-model-266bf8-kserve-route" not found 2026-07-15T21:36:59.367525Z info model Incremental push, service rate-test-model-266bf8-kserve-workload-svc.ai-tenant-e2e-shared-b-266bf8.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2026-07-15T21:36:59.468426Z info ads Push debounce stable[819] 5 for config ServiceEntry/ai-tenant-e2e-shared-b-266bf8/rate-test-model-266bf8-kserve-workload-svc.ai-tenant-e2e-shared-b-266bf8.svc.cluster.local and 3 more configs: 100.862146ms since last change, 120.755002ms since last push, full=true 2026-07-15T21:36:59.469084Z info ads XDS: Pushing Services:107 ConnectedEndpoints:4 Version:2026-07-15T21:36:59Z/438 2026-07-15T21:36:59.469631Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:1 size:44B 2026-07-15T21:36:59.469707Z info delta CDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:3 removed:1 size:507B 2026-07-15T21:36:59.469744Z info delta CDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:3 removed:1 size:507B 2026-07-15T21:36:59.469870Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:3 removed:1 size:507B 2026-07-15T21:36:59.470493Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:138 removed:0 size:33.3kB empty:1 cached:137/138 2026-07-15T21:36:59.470579Z info delta EDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:138 removed:0 size:33.3kB empty:1 cached:137/138 2026-07-15T21:36:59.470634Z info delta EDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:138 removed:0 size:33.3kB empty:1 cached:137/138 2026-07-15T21:36:59.470682Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:138 removed:0 size:33.3kB empty:1 cached:137/138 2026-07-15T21:36:59.470880Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:36:59.470989Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:36:59.471042Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-07-15T21:36:59.470894Z info delta LDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:36:59.471085Z info delta SDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:4.6kB cached:1/1 2026-07-15T21:36:59.470900Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.6kB 2026-07-15T21:36:59.471223Z info delta RDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:2.8kB cached:0/0 2026-07-15T21:36:59.471438Z info delta SDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:4.5kB cached:1/1 2026-07-15T21:36:59.471553Z info delta RDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:3.8kB cached:0/0 2026-07-15T21:36:59.471625Z info delta SDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.5kB cached:1/1 2026-07-15T21:36:59.472172Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:51.7kB cached:0/0 2026-07-15T21:36:59.472215Z info delta SDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.5kB cached:1/1 2026-07-15T21:36:59.618354Z info ads Push Status: { "pilot_duplicate_envoy_clusters": { "kuadrant-auth-service": { "proxy": "maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress", "message": "Duplicate cluster kuadrant-auth-service found while pushing CDS" } } } 2026-07-15T21:37:04.395885Z info ads Push debounce stable[820] 2 for config WasmPlugin/openshift-ingress/kuadrant-e2e-shared-a-677131 and 1 more configs: 100.411457ms since last change, 189.77611ms since last push, full=true 2026-07-15T21:37:04.406081Z info ads XDS: Pushing Services:107 ConnectedEndpoints:4 Version:2026-07-15T21:37:04Z/439 2026-07-15T21:37:04.406458Z info delta LDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:37:04.406481Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.6kB 2026-07-15T21:37:04.406529Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:37:04.406458Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:37:04.407318Z info delta ECDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:8.4kB 2026-07-15T21:37:04.407382Z info delta ECDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:8.4kB 2026-07-15T21:37:04.413387Z info delta ECDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:135.6kB 2026-07-15T21:37:08.122749Z info ads Push debounce stable[821] 4 for config Secret/premium-users-namespace/default-dockercfg-cj4pz and 3 more configs: 100.286045ms since last change, 106.698835ms since last push, full=false 2026-07-15T21:37:08.122818Z info ads XDS: Incremental Pushing ConnectedEndpoints:4 Version:2026-07-15T21:37:04Z/439 2026-07-15T21:37:09.121945Z info ads Push debounce stable[822] 4 for config Secret/maas-admin/deployer-dockercfg-g2s7x and 3 more configs: 100.375736ms since last change, 104.848838ms since last push, full=false 2026-07-15T21:37:09.122002Z info ads XDS: Incremental Pushing ConnectedEndpoints:4 Version:2026-07-15T21:37:04Z/439 2026-07-15T21:37:11.119030Z info ads Push debounce stable[823] 1 for config Secret/default/tester-regular-user-dockercfg-w4gk7: 101.070787ms since last change, 101.070702ms since last push, full=false 2026-07-15T21:37:11.119079Z info ads XDS: Incremental Pushing ConnectedEndpoints:4 Version:2026-07-15T21:37:04Z/439 2026-07-15T21:37:19.375841Z info ads Push debounce stable[824] 3 for config ServiceEntry/openshift-ingress/payload-pre-processing.openshift-ingress.svc.cluster.local and 2 more configs: 100.887767ms since last change, 183.69801ms since last push, full=true 2026-07-15T21:37:19.377216Z info ads XDS: Pushing Services:107 ConnectedEndpoints:4 Version:2026-07-15T21:37:19Z/440 2026-07-15T21:37:19.378503Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:138 removed:0 size:160.3kB cached:135/137 2026-07-15T21:37:19.378660Z info delta CDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:140 removed:0 size:160.7kB cached:135/137 2026-07-15T21:37:19.378850Z info delta CDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:140 removed:0 size:160.7kB cached:135/137 2026-07-15T21:37:19.378857Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:140 removed:0 size:160.7kB cached:135/137 2026-07-15T21:37:19.379292Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:137 removed:0 size:33.1kB empty:0 cached:135/137 2026-07-15T21:37:19.379444Z info delta EDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:137 removed:0 size:33.1kB empty:0 cached:137/137 2026-07-15T21:37:19.379603Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:37:19.379704Z info delta EDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:137 removed:0 size:33.1kB empty:0 cached:137/137 2026-07-15T21:37:19.379718Z info delta LDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:4.6kB 2026-07-15T21:37:19.380010Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:37:19.379746Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-07-15T21:37:19.379916Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:137 removed:0 size:33.1kB empty:0 cached:137/137 2026-07-15T21:37:19.380174Z info delta RDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:2.8kB cached:0/0 2026-07-15T21:37:19.380268Z info delta RDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:3.8kB cached:0/0 2026-07-15T21:37:19.380347Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:37:19.381091Z info delta ECDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:8.4kB 2026-07-15T21:37:19.381100Z info delta ECDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:8.4kB 2026-07-15T21:37:19.381486Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:51.7kB cached:0/0 2026-07-15T21:37:19.391940Z info delta ECDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:135.6kB 2026-07-15T21:37:19.619301Z info ads Push Status: { "pilot_duplicate_envoy_clusters": { "kuadrant-auth-service": { "proxy": "e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress", "message": "Duplicate cluster kuadrant-auth-service found while pushing CDS" } } } 2026-07-15T21:37:29.137888Z info model Incremental push, service maas-api-e2e-shared-a-677131.odh-ai-gateway-infra.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2026-07-15T21:37:29.274952Z info ads Push debounce stable[825] 5 for config DestinationRule/openshift-ingress/maas-api-backend-tls-e2e-shared-a-677131 and 3 more configs: 100.961467ms since last change, 144.703492ms since last push, full=true 2026-07-15T21:37:29.275566Z info ads XDS: Pushing Services:106 ConnectedEndpoints:4 Version:2026-07-15T21:37:29Z/441 2026-07-15T21:37:29.276079Z info delta CDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:3 removed:1 size:507B 2026-07-15T21:37:29.276130Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:1 size:44B 2026-07-15T21:37:29.276227Z info delta CDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:3 removed:1 size:507B 2026-07-15T21:37:29.276298Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:3 removed:1 size:507B 2026-07-15T21:37:29.276976Z info delta EDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:137 removed:0 size:33.0kB empty:1 cached:136/137 2026-07-15T21:37:29.276983Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:137 removed:0 size:33.0kB empty:1 cached:136/137 2026-07-15T21:37:29.277246Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:137 removed:0 size:33.0kB empty:1 cached:136/137 2026-07-15T21:37:29.277373Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:37:29.277478Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:37:29.277569Z info delta RDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:2.8kB cached:0/0 2026-07-15T21:37:29.277201Z info delta EDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:137 removed:0 size:33.0kB empty:1 cached:136/137 2026-07-15T21:37:29.277658Z info delta SDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.5kB cached:1/1 2026-07-15T21:37:29.277491Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:37:29.277886Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-07-15T21:37:29.277932Z info delta SDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:4.6kB cached:1/1 2026-07-15T21:37:29.277931Z info delta LDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:4.6kB 2026-07-15T21:37:29.277974Z warn constructed http route config for route https.443.default.e2e-shared-a-677131-istio-autogenerated-k8s-gateway-https.openshift-ingress on port 443 with no vhosts; Setting up a default 404 vhost 2026-07-15T21:37:29.278004Z info delta RDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:125B cached:0/0 2026-07-15T21:37:29.278039Z info delta SDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:4.5kB cached:1/1 2026-07-15T21:37:29.278823Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:51.7kB cached:0/0 2026-07-15T21:37:29.278861Z info delta SDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.5kB cached:1/1 2026-07-15T21:37:29.619363Z info ads Push Status: { "pilot_duplicate_envoy_clusters": { "kuadrant-auth-service": { "proxy": "maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress", "message": "Duplicate cluster kuadrant-auth-service found while pushing CDS" } } } 2026-07-15T21:37:32.684015Z info ads Push debounce stable[826] 1 for config EnvoyFilter/openshift-ingress/kuadrant-auth-e2e-shared-a-677131: 100.540951ms since last change, 100.540879ms since last push, full=true 2026-07-15T21:37:32.684254Z info ads XDS: Pushing Services:106 ConnectedEndpoints:4 Version:2026-07-15T21:37:32Z/442 2026-07-15T21:37:32.685141Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:137 removed:0 size:159.1kB cached:136/136 2026-07-15T21:37:32.685278Z info delta CDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:139 removed:0 size:159.6kB cached:134/136 2026-07-15T21:37:32.685434Z info delta CDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:139 removed:0 size:159.6kB cached:136/136 2026-07-15T21:37:32.685441Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:139 removed:0 size:159.6kB cached:136/136 2026-07-15T21:37:32.685828Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:136 removed:0 size:32.9kB empty:0 cached:136/136 2026-07-15T21:37:32.686085Z info delta EDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:136 removed:0 size:32.9kB empty:0 cached:136/136 2026-07-15T21:37:32.686367Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:136 removed:0 size:32.9kB empty:0 cached:136/136 2026-07-15T21:37:32.686490Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:37:32.686520Z info delta EDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:136 removed:0 size:32.9kB empty:0 cached:136/136 2026-07-15T21:37:32.686651Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:37:32.686656Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-07-15T21:37:32.686386Z info delta LDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:4.6kB 2026-07-15T21:37:32.686905Z warn constructed http route config for route https.443.default.e2e-shared-a-677131-istio-autogenerated-k8s-gateway-https.openshift-ingress on port 443 with no vhosts; Setting up a default 404 vhost 2026-07-15T21:37:32.686952Z info delta RDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:125B cached:0/0 2026-07-15T21:37:32.686908Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:37:32.687135Z info delta RDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:2.8kB cached:0/0 2026-07-15T21:37:32.687824Z info delta ECDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:8.4kB 2026-07-15T21:37:32.687962Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:51.7kB cached:0/0 2026-07-15T21:37:32.688099Z info delta ECDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:8.4kB 2026-07-15T21:37:32.694418Z info delta ECDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:135.6kB 2026-07-15T21:37:33.088261Z info ads Push debounce stable[827] 1 for config EnvoyFilter/openshift-ingress/kuadrant-ratelimiting-e2e-shared-a-677131: 100.216381ms since last change, 100.21633ms since last push, full=true 2026-07-15T21:37:33.088473Z info ads XDS: Pushing Services:106 ConnectedEndpoints:4 Version:2026-07-15T21:37:33Z/443 2026-07-15T21:37:33.089291Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:137 removed:0 size:159.1kB cached:136/136 2026-07-15T21:37:33.089615Z info delta CDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:138 removed:1 size:159.4kB cached:129/136 2026-07-15T21:37:33.089628Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:139 removed:0 size:159.6kB cached:136/136 2026-07-15T21:37:33.089969Z info delta CDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:139 removed:0 size:159.6kB cached:136/136 2026-07-15T21:37:33.090416Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:136 removed:0 size:32.9kB empty:0 cached:136/136 2026-07-15T21:37:33.090562Z info delta EDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:136 removed:0 size:32.9kB empty:0 cached:136/136 2026-07-15T21:37:33.090809Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:37:33.090953Z info delta LDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:4.6kB 2026-07-15T21:37:33.091038Z warn constructed http route config for route https.443.default.e2e-shared-a-677131-istio-autogenerated-k8s-gateway-https.openshift-ingress on port 443 with no vhosts; Setting up a default 404 vhost 2026-07-15T21:37:33.091081Z info delta RDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:125B cached:0/0 2026-07-15T21:37:33.090816Z info delta EDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:136 removed:0 size:32.9kB empty:0 cached:136/136 2026-07-15T21:37:33.091160Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:136 removed:0 size:32.9kB empty:0 cached:136/136 2026-07-15T21:37:33.091325Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:37:33.091372Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:37:33.090982Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-07-15T21:37:33.091516Z info delta RDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:2.8kB cached:0/0 2026-07-15T21:37:33.091927Z info delta ECDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:8.4kB 2026-07-15T21:37:33.092250Z info delta ECDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:8.4kB 2026-07-15T21:37:33.092443Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:51.7kB cached:0/0 2026-07-15T21:37:33.098728Z info delta ECDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:135.6kB 2026-07-15T21:37:33.296219Z info ads Push debounce stable[828] 1 for config WasmPlugin/openshift-ingress/kuadrant-e2e-shared-a-677131: 100.151796ms since last change, 100.151666ms since last push, full=true 2026-07-15T21:37:33.306264Z info ads XDS: Pushing Services:106 ConnectedEndpoints:4 Version:2026-07-15T21:37:33Z/444 2026-07-15T21:37:33.306639Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:37:33.306668Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:37:33.306725Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:37:33.306642Z info delta LDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:3.9kB 2026-07-15T21:37:33.306959Z info delta ECDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:0 removed:0 size:0B 2026-07-15T21:37:33.307377Z info delta ECDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:8.4kB 2026-07-15T21:37:33.314250Z info delta ECDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:135.6kB 2026-07-15T21:37:38.962274Z info ads Push debounce stable[829] 1 for config ConfigMap/ai-tenant-e2e-shared-a-677131/kube-root-ca.crt: 100.72176ms since last change, 100.721681ms since last push, full=false 2026-07-15T21:37:38.962332Z info ads XDS: Incremental Pushing ConnectedEndpoints:4 Version:2026-07-15T21:37:33Z/444 2026-07-15T21:37:39.874192Z info ads Push debounce stable[830] 3 for config Secret/ai-tenant-e2e-shared-a-677131/builder-dockercfg-kx256 and 2 more configs: 100.194195ms since last change, 104.281763ms since last push, full=false 2026-07-15T21:37:39.874246Z info ads XDS: Incremental Pushing ConnectedEndpoints:4 Version:2026-07-15T21:37:33Z/444 2026-07-15T21:37:40.320561Z info ads Push debounce stable[831] 1 for config Gateway/openshift-ingress/e2e-shared-a-677131-istio-autogenerated-k8s-gateway-https: 100.222137ms since last change, 100.222076ms since last push, full=true 2026-07-15T21:37:40.320675Z info ads XDS: Pushing Services:106 ConnectedEndpoints:4 Version:2026-07-15T21:37:40Z/445 2026-07-15T21:37:40.321002Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:37:40.321080Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:37:40.321099Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-07-15T21:37:40.321125Z info delta LDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:3.9kB 2026-07-15T21:37:40.321147Z warn constructed http route config for route https.443.default.e2e-shared-a-677131-istio-autogenerated-k8s-gateway-https.openshift-ingress on port 443 with no vhosts; Setting up a default 404 vhost 2026-07-15T21:37:40.321180Z info delta RDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:125B cached:0/0 2026-07-15T21:37:40.321079Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:37:40.321365Z info delta RDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:2.8kB cached:0/0 2026-07-15T21:37:40.322391Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:51.7kB cached:0/0 2026-07-15T21:37:44.462652Z info model Incremental push, service e2e-shared-a-677131-openshift-default.openshift-ingress.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2026-07-15T21:37:44.564747Z info ads Push debounce stable[832] 6 for config EnvoyFilter/openshift-ingress/e2e-shared-a-677131-authn-ssl and 4 more configs: 100.29646ms since last change, 117.399118ms since last push, full=true 2026-07-15T21:37:44.565450Z info ads XDS: Pushing Services:105 ConnectedEndpoints:4 Version:2026-07-15T21:37:44Z/446 2026-07-15T21:37:44.566489Z info delta CDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:137 removed:2 size:156.9kB cached:134/134 2026-07-15T21:37:44.566518Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:135 removed:2 size:156.5kB cached:134/134 2026-07-15T21:37:44.566857Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:137 removed:2 size:156.9kB cached:134/134 2026-07-15T21:37:44.566862Z info delta CDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:135 removed:3 size:156.5kB cached:134/134 2026-07-15T21:37:44.567176Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:136 removed:0 size:32.5kB empty:2 cached:134/136 2026-07-15T21:37:44.567183Z info delta EDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:136 removed:0 size:32.5kB empty:2 cached:134/136 2026-07-15T21:37:44.567492Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:37:44.567572Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:136 removed:0 size:32.5kB empty:2 cached:134/136 2026-07-15T21:37:44.567665Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-07-15T21:37:44.567751Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:37:44.567823Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:37:44.567865Z info delta EDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:136 removed:0 size:32.5kB empty:2 cached:134/136 2026-07-15T21:37:44.567946Z info delta RDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:2.8kB cached:0/0 2026-07-15T21:37:44.568019Z info delta SDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.5kB cached:1/1 2026-07-15T21:37:44.567757Z info delta SDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:4.6kB cached:1/1 2026-07-15T21:37:44.567966Z info delta LDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:0 removed:1 size:0B 2026-07-15T21:37:44.568336Z warn buildGatewayRoutes: no gateways for router e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress 2026-07-15T21:37:44.568378Z info delta RDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:97B cached:0/0 2026-07-15T21:37:44.568395Z warn ads proxy e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress attempted to access unauthorized certificates e2e-shared-a-677131-gw-service-tls: cross namespace secret reference requires ReferenceGrant 2026-07-15T21:37:44.568423Z info delta ECDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:0 removed:0 size:0B 2026-07-15T21:37:44.568893Z info delta ECDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:8.4kB 2026-07-15T21:37:44.568977Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:51.7kB cached:0/0 2026-07-15T21:37:44.569014Z info delta SDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.5kB cached:1/1 2026-07-15T21:37:44.575748Z info delta ECDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:135.6kB 2026-07-15T21:37:45.615637Z info ads Push debounce stable[833] 3 for config ServiceEntry/openshift-ingress/payload-processing.openshift-ingress.svc.cluster.local and 2 more configs: 100.058549ms since last change, 194.666194ms since last push, full=true 2026-07-15T21:37:45.616956Z info ads XDS: Pushing Services:105 ConnectedEndpoints:4 Version:2026-07-15T21:37:45Z/447 2026-07-15T21:37:45.618244Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:135 removed:0 size:156.5kB cached:132/134 2026-07-15T21:37:45.618340Z info delta CDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:135 removed:0 size:156.5kB cached:133/134 2026-07-15T21:37:45.618377Z info delta CDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:137 removed:0 size:156.9kB cached:132/134 2026-07-15T21:37:45.618428Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:137 removed:0 size:156.9kB cached:132/134 2026-07-15T21:37:45.619070Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:134 removed:0 size:32.3kB empty:0 cached:132/134 2026-07-15T21:37:45.619266Z info delta EDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:134 removed:0 size:32.3kB empty:0 cached:133/134 2026-07-15T21:37:45.619442Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:37:45.619541Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.6kB 2026-07-15T21:37:45.619678Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:134 removed:0 size:32.3kB empty:0 cached:134/134 2026-07-15T21:37:45.619272Z info delta EDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:134 removed:0 size:32.3kB empty:0 cached:134/134 2026-07-15T21:37:45.619936Z info delta LDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:0 removed:0 size:0B 2026-07-15T21:37:45.619973Z warn buildGatewayRoutes: no gateways for router e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress 2026-07-15T21:37:45.619976Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:37:45.619561Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-07-15T21:37:45.620007Z info delta RDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:1 removed:0 size:97B cached:0/0 2026-07-15T21:37:45.619866Z info delta RDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:3.8kB cached:0/0 2026-07-15T21:37:45.620046Z info delta ECDS: PUSH for node:e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress resources:0 removed:0 size:0B 2026-07-15T21:37:45.620903Z info delta ECDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:8.4kB 2026-07-15T21:37:45.621360Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:51.7kB cached:0/0 2026-07-15T21:37:45.632583Z info delta ECDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:135.6kB 2026-07-15T21:37:49.486488Z info delta ADS: "10.134.0.64:38606" e2e-shared-a-677131-openshift-default-784d56455-tgbzp.openshift-ingress-28 terminated 2026-07-15T21:37:49.619091Z info ads Push Status: { "pilot_duplicate_envoy_clusters": { "kuadrant-auth-service": { "proxy": "maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress", "message": "Duplicate cluster kuadrant-auth-service found while pushing CDS" } } } 2026-07-15T21:37:55.428222Z info model Incremental push, service maas-api-e2e-shared-b-266bf8.odh-ai-gateway-infra.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2026-07-15T21:37:55.559574Z info ads Push debounce stable[834] 5 for config DestinationRule/openshift-ingress/maas-api-backend-tls-e2e-shared-b-266bf8 and 3 more configs: 101.023222ms since last change, 144.930834ms since last push, full=true 2026-07-15T21:37:55.560202Z info ads XDS: Pushing Services:104 ConnectedEndpoints:3 Version:2026-07-15T21:37:55Z/448 2026-07-15T21:37:55.560640Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:3 removed:1 size:507B 2026-07-15T21:37:55.560743Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:1 size:44B 2026-07-15T21:37:55.560788Z info delta CDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:3 removed:1 size:507B 2026-07-15T21:37:55.561421Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:134 removed:0 size:32.2kB empty:1 cached:133/134 2026-07-15T21:37:55.561488Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:134 removed:0 size:32.2kB empty:1 cached:133/134 2026-07-15T21:37:55.561576Z info delta EDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:134 removed:0 size:32.2kB empty:1 cached:133/134 2026-07-15T21:37:55.561764Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:37:55.561903Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.6kB 2026-07-15T21:37:55.561979Z warn constructed http route config for route https.443.default.e2e-shared-b-266bf8-istio-autogenerated-k8s-gateway-https.openshift-ingress on port 443 with no vhosts; Setting up a default 404 vhost 2026-07-15T21:37:55.562016Z info delta RDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:125B cached:0/0 2026-07-15T21:37:55.561785Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:37:55.562055Z info delta SDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.5kB cached:1/1 2026-07-15T21:37:55.561916Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-07-15T21:37:55.562412Z info delta SDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:4.6kB cached:1/1 2026-07-15T21:37:55.563146Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:51.7kB cached:0/0 2026-07-15T21:37:55.563192Z info delta SDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.5kB cached:1/1 2026-07-15T21:37:58.986598Z info ads Push debounce stable[835] 2 for config EnvoyFilter/openshift-ingress/kuadrant-auth-e2e-shared-b-266bf8 and 1 more configs: 100.055521ms since last change, 200.059499ms since last push, full=true 2026-07-15T21:37:58.986841Z info ads XDS: Pushing Services:104 ConnectedEndpoints:3 Version:2026-07-15T21:37:58Z/449 2026-07-15T21:37:58.988667Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:134 removed:0 size:155.3kB cached:133/133 2026-07-15T21:37:58.989110Z info delta CDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:135 removed:1 size:155.6kB cached:129/133 2026-07-15T21:37:58.990149Z info delta EDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:133 removed:0 size:32.1kB empty:0 cached:133/133 2026-07-15T21:37:58.990553Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:4.6kB 2026-07-15T21:37:58.990608Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:133 removed:0 size:32.1kB empty:0 cached:133/133 2026-07-15T21:37:58.990993Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:37:58.991158Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-07-15T21:37:58.990635Z warn constructed http route config for route https.443.default.e2e-shared-b-266bf8-istio-autogenerated-k8s-gateway-https.openshift-ingress on port 443 with no vhosts; Setting up a default 404 vhost 2026-07-15T21:37:58.991338Z info delta RDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:125B cached:0/0 2026-07-15T21:37:58.992502Z info delta ECDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:8.4kB 2026-07-15T21:37:58.993011Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:136 removed:0 size:155.8kB cached:133/133 2026-07-15T21:37:58.994566Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:133 removed:0 size:32.1kB empty:0 cached:133/133 2026-07-15T21:37:58.995512Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:37:58.996444Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:51.7kB cached:0/0 2026-07-15T21:37:59.003447Z info delta ECDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:135.6kB 2026-07-15T21:37:59.097989Z info ads Push debounce stable[836] 1 for config WasmPlugin/openshift-ingress/kuadrant-e2e-shared-b-266bf8: 100.042508ms since last change, 100.042437ms since last push, full=true 2026-07-15T21:37:59.107847Z info ads XDS: Pushing Services:104 ConnectedEndpoints:3 Version:2026-07-15T21:37:59Z/450 2026-07-15T21:37:59.108172Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:3.9kB 2026-07-15T21:37:59.108183Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:37:59.108217Z info delta ECDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:0 removed:0 size:0B 2026-07-15T21:37:59.108221Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.1kB 2026-07-15T21:37:59.115402Z info delta ECDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:135.6kB 2026-07-15T21:37:59.302181Z info ads Push debounce stable[837] 3 for config ServiceEntry/openshift-ingress/payload-processing.openshift-ingress.svc.cluster.local and 2 more configs: 100.772917ms since last change, 172.920363ms since last push, full=true 2026-07-15T21:37:59.303956Z info ads XDS: Pushing Services:104 ConnectedEndpoints:3 Version:2026-07-15T21:37:59Z/451 2026-07-15T21:37:59.305190Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:134 removed:0 size:155.3kB cached:131/133 2026-07-15T21:37:59.305430Z info delta CDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:135 removed:0 size:155.6kB cached:131/133 2026-07-15T21:37:59.305452Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:136 removed:0 size:155.8kB cached:131/133 2026-07-15T21:37:59.306180Z info delta EDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:133 removed:0 size:32.1kB empty:0 cached:133/133 2026-07-15T21:37:59.306191Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:133 removed:0 size:32.1kB empty:0 cached:133/133 2026-07-15T21:37:59.306190Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:133 removed:0 size:32.1kB empty:0 cached:131/133 2026-07-15T21:37:59.306452Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:3.9kB 2026-07-15T21:37:59.306506Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:37:59.306524Z warn constructed http route config for route https.443.default.e2e-shared-b-266bf8-istio-autogenerated-k8s-gateway-https.openshift-ingress on port 443 with no vhosts; Setting up a default 404 vhost 2026-07-15T21:37:59.306560Z info delta RDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:125B cached:0/0 2026-07-15T21:37:59.306589Z info delta ECDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:0 removed:0 size:0B 2026-07-15T21:37:59.306653Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-07-15T21:37:59.306726Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.5kB 2026-07-15T21:37:59.308269Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:52.6kB cached:0/0 2026-07-15T21:37:59.320299Z info delta ECDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:135.6kB 2026-07-15T21:37:59.619330Z info ads Push Status: { "pilot_duplicate_envoy_clusters": { "kuadrant-auth-service": { "proxy": "maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress", "message": "Duplicate cluster kuadrant-auth-service found while pushing CDS" } } } 2026-07-15T21:38:05.321078Z info ads Push debounce stable[838] 3 for config Secret/ai-tenant-e2e-shared-b-266bf8/default-dockercfg-mgwpw and 2 more configs: 100.208017ms since last change, 100.901607ms since last push, full=false 2026-07-15T21:38:05.321146Z info ads XDS: Incremental Pushing ConnectedEndpoints:3 Version:2026-07-15T21:37:59Z/451 2026-07-15T21:38:05.501087Z info ads Push debounce stable[839] 1 for config ConfigMap/ai-tenant-e2e-shared-b-266bf8/kube-root-ca.crt: 100.739281ms since last change, 100.739201ms since last push, full=false 2026-07-15T21:38:05.501153Z info ads XDS: Incremental Pushing ConnectedEndpoints:3 Version:2026-07-15T21:37:59Z/451 2026-07-15T21:38:06.733198Z info ads Push debounce stable[840] 1 for config Gateway/openshift-ingress/e2e-shared-b-266bf8-istio-autogenerated-k8s-gateway-https: 100.222544ms since last change, 100.222492ms since last push, full=true 2026-07-15T21:38:06.733315Z info ads XDS: Pushing Services:104 ConnectedEndpoints:3 Version:2026-07-15T21:38:06Z/452 2026-07-15T21:38:06.733661Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:3.9kB 2026-07-15T21:38:06.733692Z warn constructed http route config for route https.443.default.e2e-shared-b-266bf8-istio-autogenerated-k8s-gateway-https.openshift-ingress on port 443 with no vhosts; Setting up a default 404 vhost 2026-07-15T21:38:06.733689Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.5kB 2026-07-15T21:38:06.733721Z info delta RDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:125B cached:0/0 2026-07-15T21:38:06.733856Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:38:06.733979Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-07-15T21:38:06.734907Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:52.6kB cached:0/0 2026-07-15T21:38:10.668813Z info model Incremental push, service e2e-shared-b-266bf8-openshift-default.openshift-ingress.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2026-07-15T21:38:10.771011Z info ads Push debounce stable[841] 6 for config Gateway/openshift-ingress/e2e-shared-b-266bf8-istio-autogenerated-k8s-gateway-https and 4 more configs: 100.22929ms since last change, 128.568456ms since last push, full=true 2026-07-15T21:38:10.771629Z info ads XDS: Pushing Services:103 ConnectedEndpoints:3 Version:2026-07-15T21:38:10Z/453 2026-07-15T21:38:10.772596Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:134 removed:2 size:153.1kB cached:131/131 2026-07-15T21:38:10.772608Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:132 removed:2 size:152.7kB cached:131/131 2026-07-15T21:38:10.772608Z info delta CDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:132 removed:3 size:152.7kB cached:131/131 2026-07-15T21:38:10.773518Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:133 removed:0 size:31.7kB empty:2 cached:131/133 2026-07-15T21:38:10.773519Z info delta EDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:133 removed:0 size:31.7kB empty:2 cached:131/133 2026-07-15T21:38:10.773867Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.5kB 2026-07-15T21:38:10.773903Z info delta LDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:0 removed:1 size:0B 2026-07-15T21:38:10.773918Z warn buildGatewayRoutes: no gateways for router e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress 2026-07-15T21:38:10.773941Z info delta RDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:1 removed:0 size:97B cached:0/0 2026-07-15T21:38:10.773960Z warn ads proxy e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress attempted to access unauthorized certificates e2e-shared-b-266bf8-gw-service-tls: cross namespace secret reference requires ReferenceGrant 2026-07-15T21:38:10.773983Z info delta ECDS: PUSH for node:e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress resources:0 removed:0 size:0B 2026-07-15T21:38:10.773523Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:133 removed:0 size:31.7kB empty:2 cached:131/133 2026-07-15T21:38:10.774258Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:7.2kB 2026-07-15T21:38:10.774390Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-07-15T21:38:10.774430Z info delta SDS: PUSH for node:data-science-gateway-data-science-gateway-class-6576859cf46sl55.openshift-ingress resources:1 removed:0 size:4.6kB cached:1/1 2026-07-15T21:38:10.775413Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:52.6kB cached:0/0 2026-07-15T21:38:10.775461Z info delta SDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:4.5kB cached:1/1 2026-07-15T21:38:10.787457Z info delta ECDS: PUSH for node:maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress resources:1 removed:0 size:135.6kB 2026-07-15T21:38:15.690310Z info delta ADS: "10.134.0.66:54598" e2e-shared-b-266bf8-openshift-default-7fb9bb959-9t8lt.openshift-ingress-29 terminated 2026-07-15T21:38:19.618509Z info ads Push Status: { "pilot_duplicate_envoy_clusters": { "kuadrant-auth-service": { "proxy": "maas-default-gateway-openshift-default-56654dcf7-blfs5.openshift-ingress", "message": "Duplicate cluster kuadrant-auth-service found while pushing CDS" } } }