2026-11-01T04:33:29.098403Z info delta LDS: PUSH for node:e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress resources:1 removed:0 size:3.9kB 2026-11-01T04:33:29.098429Z warn constructed http route config for route https.443.default.e2e-shared-b-w2-fb1c1f~istio-autogenerated-k8s-gateway~https.openshift-ingress on port 443 with no vhosts; Setting up a default 404 vhost 2026-11-01T04:33:29.098457Z info delta RDS: PUSH for node:e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress resources:1 removed:0 size:128B cached:0/0 2026-11-01T04:33:29.099122Z info delta LDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:176.3kB 2026-11-01T04:33:29.099144Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:310.2kB 2026-11-01T04:33:29.100725Z info delta RDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:63.3kB cached:0/0 2026-11-01T04:33:29.102379Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:111.6kB cached:0/0 2026-11-01T04:33:30.677782Z info ads Push debounce stable[2545] 1 for config ServiceEntry/odh-ai-gateway-infra/maas-api-metrics.odh-ai-gateway-infra.svc.cluster.local: 100.836123ms since last change, 100.836054ms since last push, full=true 2026-11-01T04:33:30.678475Z info ads XDS: Pushing Services:119 ConnectedEndpoints:5 Version:2026-11-01T04:33:30Z/1723 2026-11-01T04:33:30.679145Z info delta CDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:2 removed:0 size:1.4kB cached:0/1 2026-11-01T04:33:30.679196Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:2 removed:0 size:1.4kB cached:1/1 2026-11-01T04:33:30.679217Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:8 removed:0 size:3.0kB cached:0/1 2026-11-01T04:33:30.679274Z info delta EDS: PUSH INC for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:344B empty:0 cached:1/1 2026-11-01T04:33:30.679277Z info delta EDS: PUSH INC for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:1 removed:0 size:344B empty:0 cached:0/1 2026-11-01T04:33:30.679434Z info delta CDS: PUSH for node:e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress resources:3 removed:0 size:1.7kB cached:0/1 2026-11-01T04:33:30.679442Z warn no virtual service bound to gateway: openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https 2026-11-01T04:33:30.679450Z warn gateway openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https:443 listener missed network filter 2026-11-01T04:33:30.679455Z info delta CDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:8 removed:0 size:3.1kB cached:0/1 2026-11-01T04:33:30.679463Z info gateway omitting listener "0.0.0.0_443" due to: must have more than 0 chains in listener "0.0.0.0_443" 2026-11-01T04:33:30.679487Z info delta LDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:0 removed:0 size:0B 2026-11-01T04:33:30.679409Z info delta EDS: PUSH INC for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:344B empty:0 cached:1/1 2026-11-01T04:33:30.679507Z info delta EDS: PUSH INC for node:e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress resources:1 removed:0 size:344B empty:0 cached:1/1 2026-11-01T04:33:30.679519Z info delta EDS: PUSH INC for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:344B empty:0 cached:1/1 2026-11-01T04:33:30.679872Z info delta LDS: PUSH for node:e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress resources:1 removed:0 size:3.9kB 2026-11-01T04:33:30.679555Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:7.2kB 2026-11-01T04:33:30.679896Z warn constructed http route config for route https.443.default.e2e-shared-b-w2-fb1c1f~istio-autogenerated-k8s-gateway~https.openshift-ingress on port 443 with no vhosts; Setting up a default 404 vhost 2026-11-01T04:33:30.679922Z info delta RDS: PUSH for node:e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress resources:1 removed:0 size:128B cached:0/0 2026-11-01T04:33:30.680006Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-11-01T04:33:30.680576Z info delta LDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:176.3kB 2026-11-01T04:33:30.680608Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:310.2kB 2026-11-01T04:33:30.682269Z info delta RDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:63.3kB cached:0/0 2026-11-01T04:33:30.683905Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:111.6kB cached:0/0 2026-11-01T04:33:31.053352Z info ads Push debounce stable[2546] 1 for config ConfigMap/ai-tenant-e2e-shared-b-w2-fb1c1f/kube-root-ca.crt: 100.196075ms since last change, 100.195992ms since last push, full=false 2026-11-01T04:33:31.053406Z info ads XDS: Incremental Pushing ConnectedEndpoints:5 Version:2026-11-01T04:33:30Z/1723 2026-11-01T04:33:31.368821Z info ads Push debounce stable[2547] 3 for config Secret/ai-tenant-e2e-shared-b-w2-fb1c1f/deployer-dockercfg-c84tj and 2 more configs: 100.255638ms since last change, 104.184673ms since last push, full=false 2026-11-01T04:33:31.368873Z info ads XDS: Incremental Pushing ConnectedEndpoints:5 Version:2026-11-01T04:33:30Z/1723 2026-11-01T04:33:32.053683Z info ads Push debounce stable[2548] 1 for config Gateway/openshift-ingress/e2e-shared-b-w2-fb1c1f~istio-autogenerated-k8s-gateway~https: 100.218678ms since last change, 100.218583ms since last push, full=true 2026-11-01T04:33:32.053819Z info ads XDS: Pushing Services:119 ConnectedEndpoints:5 Version:2026-11-01T04:33:32Z/1724 2026-11-01T04:33:32.053966Z warn no virtual service bound to gateway: openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https 2026-11-01T04:33:32.053985Z warn gateway openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https:443 listener missed network filter 2026-11-01T04:33:32.054002Z info gateway omitting listener "0.0.0.0_443" due to: must have more than 0 chains in listener "0.0.0.0_443" 2026-11-01T04:33:32.054089Z info delta LDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:0 removed:0 size:0B 2026-11-01T04:33:32.054229Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:7.2kB 2026-11-01T04:33:32.054348Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-11-01T04:33:32.054229Z info delta LDS: PUSH for node:e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress resources:1 removed:0 size:3.9kB 2026-11-01T04:33:32.054449Z warn constructed http route config for route https.443.default.e2e-shared-b-w2-fb1c1f~istio-autogenerated-k8s-gateway~https.openshift-ingress on port 443 with no vhosts; Setting up a default 404 vhost 2026-11-01T04:33:32.054484Z info delta RDS: PUSH for node:e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress resources:1 removed:0 size:128B cached:0/0 2026-11-01T04:33:32.054658Z info delta LDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:176.3kB 2026-11-01T04:33:32.054817Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:310.2kB 2026-11-01T04:33:32.056017Z info delta RDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:63.3kB cached:0/0 2026-11-01T04:33:32.057423Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:111.6kB cached:0/0 2026-11-01T04:33:32.291164Z info model Incremental push, service e2e-shared-b-w2-fb1c1f-openshift-default.openshift-ingress.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2026-11-01T04:33:32.391681Z info ads Push debounce stable[2549] 6 for config Gateway/openshift-ingress/e2e-shared-b-w2-fb1c1f~istio-autogenerated-k8s-gateway~https and 4 more configs: 100.480663ms since last change, 122.412307ms since last push, full=true 2026-11-01T04:33:32.392448Z info ads XDS: Pushing Services:118 ConnectedEndpoints:5 Version:2026-11-01T04:33:32Z/1725 2026-11-01T04:33:32.393979Z info delta CDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:156 removed:2 size:175.9kB cached:149/149 2026-11-01T04:33:32.393994Z info delta CDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:150 removed:2 size:174.1kB cached:149/149 2026-11-01T04:33:32.394027Z info delta CDS: PUSH for node:e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress resources:150 removed:3 size:174.1kB cached:149/149 2026-11-01T04:33:32.393993Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:150 removed:2 size:174.1kB cached:149/149 2026-11-01T04:33:32.394384Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:156 removed:2 size:175.8kB cached:149/149 2026-11-01T04:33:32.394981Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:151 removed:0 size:36.8kB empty:2 cached:149/151 2026-11-01T04:33:32.395258Z info delta EDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:151 removed:0 size:36.8kB empty:2 cached:149/151 2026-11-01T04:33:32.395285Z info delta EDS: PUSH for node:e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress resources:151 removed:0 size:36.8kB empty:2 cached:149/151 2026-11-01T04:33:32.395103Z info delta EDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:151 removed:0 size:36.8kB empty:2 cached:149/151 2026-11-01T04:33:32.395335Z info delta LDS: PUSH for node:e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress resources:0 removed:1 size:0B 2026-11-01T04:33:32.395349Z warn no virtual service bound to gateway: openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https 2026-11-01T04:33:32.395351Z warn buildGatewayRoutes: no gateways for router e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress 2026-11-01T04:33:32.395356Z warn gateway openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https:443 listener missed network filter 2026-11-01T04:33:32.395375Z info delta RDS: PUSH for node:e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress resources:1 removed:0 size:100B cached:0/0 2026-11-01T04:33:32.395379Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:151 removed:0 size:36.8kB empty:2 cached:149/151 2026-11-01T04:33:32.395382Z info gateway omitting listener "0.0.0.0_443" due to: must have more than 0 chains in listener "0.0.0.0_443" 2026-11-01T04:33:32.395411Z info delta LDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:0 removed:0 size:0B 2026-11-01T04:33:32.395461Z warn ads proxy e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress attempted to access unauthorized certificates e2e-shared-b-w2-fb1c1f-gw-service-tls: cross namespace secret reference requires ReferenceGrant 2026-11-01T04:33:32.395719Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:7.2kB 2026-11-01T04:33:32.395820Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-11-01T04:33:32.395858Z info delta SDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:4.6kB cached:1/1 2026-11-01T04:33:32.396234Z info delta LDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:176.3kB 2026-11-01T04:33:32.396317Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:310.2kB 2026-11-01T04:33:32.397733Z info delta RDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:63.3kB cached:0/0 2026-11-01T04:33:32.397767Z info delta SDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:4.5kB cached:1/1 2026-11-01T04:33:32.398236Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:111.6kB cached:0/0 2026-11-01T04:33:32.398360Z info delta SDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:4.5kB cached:1/1 2026-11-01T04:33:33.354884Z info gateway reconciling gateway with controller openshift.io/gateway-controller/v1 gateway=openshift-ingress/e2e-worker-w2-865066 2026-11-01T04:33:33.354917Z info gateway reconciling gateway=openshift-ingress/e2e-worker-w2-865066 2026-11-01T04:33:33.637175Z info gateway gateway updated gateway=openshift-ingress/e2e-worker-w2-865066 2026-11-01T04:33:35.379621Z info ads Push Status: { "pilot_duplicate_envoy_clusters": { "kuadrant-auth-service": { "proxy": "maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress", "message": "Duplicate cluster kuadrant-auth-service found while pushing CDS" } } } 2026-11-01T04:33:36.031073Z info ads Push debounce stable[2550] 1 for config ServiceEntry/odh-ai-gateway-infra/maas-api-metrics.odh-ai-gateway-infra.svc.cluster.local: 100.512821ms since last change, 100.512745ms since last push, full=true 2026-11-01T04:33:36.031719Z info ads XDS: Pushing Services:118 ConnectedEndpoints:5 Version:2026-11-01T04:33:36Z/1726 2026-11-01T04:33:36.032418Z info delta CDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:2 removed:0 size:1.4kB cached:0/1 2026-11-01T04:33:36.032491Z info delta CDS: PUSH for node:e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress resources:2 removed:0 size:1.4kB cached:1/1 2026-11-01T04:33:36.032547Z info delta EDS: PUSH INC for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:1 removed:0 size:344B empty:0 cached:0/1 2026-11-01T04:33:36.032574Z info delta EDS: PUSH INC for node:e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress resources:1 removed:0 size:344B empty:0 cached:1/1 2026-11-01T04:33:36.032657Z info delta LDS: PUSH for node:e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress resources:0 removed:0 size:0B 2026-11-01T04:33:36.032575Z warn no virtual service bound to gateway: openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https 2026-11-01T04:33:36.032678Z warn gateway openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https:443 listener missed network filter 2026-11-01T04:33:36.032420Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:2 removed:0 size:1.4kB cached:1/1 2026-11-01T04:33:36.032681Z warn buildGatewayRoutes: no gateways for router e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress 2026-11-01T04:33:36.032735Z info delta RDS: PUSH for node:e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress resources:1 removed:0 size:100B cached:0/0 2026-11-01T04:33:36.032579Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:8 removed:0 size:3.0kB cached:0/1 2026-11-01T04:33:36.032702Z info gateway omitting listener "0.0.0.0_443" due to: must have more than 0 chains in listener "0.0.0.0_443" 2026-11-01T04:33:36.033124Z info delta LDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:0 removed:0 size:0B 2026-11-01T04:33:36.033234Z info delta CDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:8 removed:0 size:3.1kB cached:0/1 2026-11-01T04:33:36.037178Z info delta EDS: PUSH INC for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:344B empty:0 cached:1/1 2026-11-01T04:33:36.033398Z info delta EDS: PUSH INC for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:344B empty:0 cached:1/1 2026-11-01T04:33:36.033493Z info delta EDS: PUSH INC for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:344B empty:0 cached:1/1 2026-11-01T04:33:36.038195Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:7.2kB 2026-11-01T04:33:36.038373Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-11-01T04:33:36.038510Z info delta LDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:176.3kB 2026-11-01T04:33:36.039368Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:310.2kB 2026-11-01T04:33:36.040498Z info delta RDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:63.3kB cached:0/0 2026-11-01T04:33:36.043325Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:111.6kB cached:0/0 2026-11-01T04:33:37.913587Z info ads Push debounce stable[2551] 4 for config ServiceEntry/e2e-models-e2e-worker-w2-865066/distinct-2-w2-865066-kserve-workload-svc.e2e-models-e2e-worker-w2-865066.svc.cluster.local and 3 more configs: 100.392807ms since last change, 135.5733ms since last push, full=false 2026-11-01T04:33:37.913662Z info ads XDS: Incremental Pushing ConnectedEndpoints:5 Version:2026-11-01T04:33:36Z/1726 2026-11-01T04:33:39.204419Z info gateway reconciling gateway with controller openshift.io/gateway-controller/v1 gateway=openshift-ingress/openshift-ai-inference 2026-11-01T04:33:39.204458Z info gateway reconciling gateway=openshift-ingress/openshift-ai-inference 2026-11-01T04:33:39.481009Z info gateway gateway updated gateway=openshift-ingress/openshift-ai-inference 2026-11-01T04:33:39.481069Z info gateway reconciling gateway with controller openshift.io/gateway-controller/v1 gateway=openshift-ingress/data-science-gateway 2026-11-01T04:33:39.481085Z info gateway reconciling gateway=openshift-ingress/data-science-gateway 2026-11-01T04:33:39.756808Z info gateway gateway updated gateway=openshift-ingress/data-science-gateway 2026-11-01T04:33:39.756852Z info gateway reconciling gateway with controller openshift.io/gateway-controller/v1 gateway=openshift-ingress/maas-default-gateway 2026-11-01T04:33:39.756867Z info gateway reconciling gateway=openshift-ingress/maas-default-gateway 2026-11-01T04:33:40.035124Z info gateway gateway updated gateway=openshift-ingress/maas-default-gateway 2026-11-01T04:33:41.096653Z info ads Push debounce stable[2552] 1 for config ServiceEntry/odh-ai-gateway-infra/maas-api-metrics.odh-ai-gateway-infra.svc.cluster.local: 100.61553ms since last change, 100.615461ms since last push, full=true 2026-11-01T04:33:41.097321Z info ads XDS: Pushing Services:118 ConnectedEndpoints:5 Version:2026-11-01T04:33:41Z/1727 2026-11-01T04:33:41.098069Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:2 removed:0 size:1.4kB cached:0/1 2026-11-01T04:33:41.098126Z info delta CDS: PUSH for node:e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress resources:2 removed:0 size:1.4kB cached:1/1 2026-11-01T04:33:41.098205Z info delta EDS: PUSH INC for node:e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress resources:1 removed:0 size:344B empty:0 cached:1/1 2026-11-01T04:33:41.098210Z info delta EDS: PUSH INC for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:344B empty:0 cached:0/1 2026-11-01T04:33:41.098246Z info delta LDS: PUSH for node:e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress resources:0 removed:0 size:0B 2026-11-01T04:33:41.098261Z warn buildGatewayRoutes: no gateways for router e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress 2026-11-01T04:33:41.098277Z info delta CDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:8 removed:0 size:3.1kB cached:0/1 2026-11-01T04:33:41.098073Z info delta CDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:2 removed:0 size:1.4kB cached:0/1 2026-11-01T04:33:41.098377Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:8 removed:0 size:3.0kB cached:0/1 2026-11-01T04:33:41.098406Z info delta EDS: PUSH INC for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:344B empty:0 cached:1/1 2026-11-01T04:33:41.098442Z info delta EDS: PUSH INC for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:344B empty:0 cached:1/1 2026-11-01T04:33:41.098456Z info delta EDS: PUSH INC for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:1 removed:0 size:344B empty:0 cached:1/1 2026-11-01T04:33:41.098486Z warn no virtual service bound to gateway: openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https 2026-11-01T04:33:41.098284Z info delta RDS: PUSH for node:e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress resources:1 removed:0 size:100B cached:0/0 2026-11-01T04:33:41.098496Z warn gateway openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https:443 listener missed network filter 2026-11-01T04:33:41.098515Z info gateway omitting listener "0.0.0.0_443" due to: must have more than 0 chains in listener "0.0.0.0_443" 2026-11-01T04:33:41.098543Z info delta LDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:0 removed:0 size:0B 2026-11-01T04:33:41.098547Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:7.2kB 2026-11-01T04:33:41.098637Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-11-01T04:33:41.099257Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:310.2kB 2026-11-01T04:33:41.099288Z info delta LDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:176.3kB 2026-11-01T04:33:41.100998Z info delta RDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:63.3kB cached:0/0 2026-11-01T04:33:41.102441Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:111.6kB cached:0/0 2026-11-01T04:33:42.679188Z info delta ADS: "10.134.0.97:53300" e2e-shared-b-w2-fb1c1f-openshift-default-6cb7dc57c9-dqfkj.openshift-ingress-104 terminated 2026-11-01T04:33:44.876543Z info ads Push debounce stable[2553] 1 for config ServiceEntry/openshift-ingress/payload-processing-e2e-worker-w2-865066.openshift-ingress.svc.cluster.local: 100.538467ms since last change, 100.538385ms since last push, full=false 2026-11-01T04:33:44.876596Z info ads XDS: Incremental Pushing ConnectedEndpoints:4 Version:2026-11-01T04:33:41Z/1727 2026-11-01T04:33:44.979226Z info ads Push debounce stable[2554] 1 for config ServiceEntry/openshift-ingress/payload-pre-processing-e2e-worker-w2-865066.openshift-ingress.svc.cluster.local: 100.397098ms since last change, 100.397028ms since last push, full=false 2026-11-01T04:33:44.979292Z info ads XDS: Incremental Pushing ConnectedEndpoints:4 Version:2026-11-01T04:33:41Z/1727 2026-11-01T04:33:45.056938Z info model Incremental push, service payload-processing-e2e-worker-w2-865066.openshift-ingress.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2026-11-01T04:33:45.165414Z info model Incremental push, service payload-pre-processing-e2e-worker-w2-865066.openshift-ingress.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2026-11-01T04:33:45.265511Z info ads Push debounce stable[2555] 4 for config ServiceEntry/openshift-ingress/payload-processing-e2e-worker-w2-865066.openshift-ingress.svc.cluster.local and 1 more configs: 100.050125ms since last change, 216.362836ms since last push, full=true 2026-11-01T04:33:45.266167Z info ads XDS: Pushing Services:116 ConnectedEndpoints:4 Version:2026-11-01T04:33:45Z/1728 2026-11-01T04:33:45.266693Z info delta CDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:1 removed:2 size:44B 2026-11-01T04:33:45.266758Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:2 size:44B 2026-11-01T04:33:45.266772Z info delta EDS: PUSH INC for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:0 removed:2 size:0B empty:0 cached:0/0 2026-11-01T04:33:45.266797Z warn no virtual service bound to gateway: openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https 2026-11-01T04:33:45.266865Z warn gateway openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https:443 listener missed network filter 2026-11-01T04:33:45.266887Z info gateway omitting listener "0.0.0.0_443" due to: must have more than 0 chains in listener "0.0.0.0_443" 2026-11-01T04:33:45.266912Z info delta LDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:0 removed:0 size:0B 2026-11-01T04:33:45.266835Z info delta EDS: PUSH INC for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:0 removed:2 size:0B empty:0 cached:0/0 2026-11-01T04:33:45.267015Z info delta CDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:7 removed:2 size:1.8kB 2026-11-01T04:33:45.267106Z info delta EDS: PUSH INC for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:0 removed:2 size:0B empty:0 cached:0/0 2026-11-01T04:33:45.267185Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:7 removed:2 size:1.7kB 2026-11-01T04:33:45.267237Z info delta EDS: PUSH INC for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:0 removed:2 size:0B empty:0 cached:0/0 2026-11-01T04:33:45.267252Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:7.2kB 2026-11-01T04:33:45.267349Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-11-01T04:33:45.267782Z info delta LDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:176.3kB 2026-11-01T04:33:45.268002Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:310.2kB 2026-11-01T04:33:45.269369Z info delta RDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:63.3kB cached:0/0 2026-11-01T04:33:45.270596Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:111.6kB cached:0/0 2026-11-01T04:33:45.379335Z info ads Push Status: { "pilot_duplicate_envoy_clusters": { "kuadrant-auth-service": { "proxy": "maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress", "message": "Duplicate cluster kuadrant-auth-service found while pushing CDS" } } } 2026-11-01T04:33:45.662088Z info ads Push debounce stable[2556] 1 for config Secret/openshift-ingress/payload-processing-e2e-worker-w2-865066-dockercfg-qr8g4: 100.114451ms since last change, 100.114373ms since last push, full=false 2026-11-01T04:33:45.662139Z info ads XDS: Incremental Pushing ConnectedEndpoints:4 Version:2026-11-01T04:33:45Z/1728 2026-11-01T04:33:46.106955Z info ads Push debounce stable[2557] 1 for config EnvoyFilter/openshift-ingress/payload-processing-e2e-worker-w2-865066: 100.22693ms since last change, 100.226852ms since last push, full=true 2026-11-01T04:33:46.107285Z info ads XDS: Pushing Services:116 ConnectedEndpoints:4 Version:2026-11-01T04:33:46Z/1729 2026-11-01T04:33:46.110135Z info delta CDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:148 removed:0 size:171.6kB cached:146/147 2026-11-01T04:33:46.110135Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:148 removed:0 size:171.6kB cached:143/147 2026-11-01T04:33:46.110374Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:154 removed:0 size:173.2kB cached:143/147 2026-11-01T04:33:46.110815Z info delta EDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:147 removed:0 size:35.3kB empty:0 cached:147/147 2026-11-01T04:33:46.110857Z warn no virtual service bound to gateway: openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https 2026-11-01T04:33:46.110863Z warn gateway openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https:443 listener missed network filter 2026-11-01T04:33:46.110883Z info gateway omitting listener "0.0.0.0_443" due to: must have more than 0 chains in listener "0.0.0.0_443" 2026-11-01T04:33:46.110910Z info delta LDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:0 removed:0 size:0B 2026-11-01T04:33:46.111176Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:147 removed:0 size:35.3kB empty:0 cached:147/147 2026-11-01T04:33:46.111214Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:147 removed:0 size:35.3kB empty:0 cached:147/147 2026-11-01T04:33:46.111536Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:7.2kB 2026-11-01T04:33:46.111649Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-11-01T04:33:46.112026Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:310.2kB 2026-11-01T04:33:46.112269Z info delta CDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:152 removed:2 size:172.3kB cached:0/147 2026-11-01T04:33:46.112894Z info delta EDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:147 removed:0 size:35.3kB empty:0 cached:147/147 2026-11-01T04:33:46.113499Z info delta LDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:175.8kB 2026-11-01T04:33:46.114352Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:111.6kB cached:0/0 2026-11-01T04:33:46.114802Z info delta RDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:63.3kB cached:0/0 2026-11-01T04:33:46.399398Z info ads Push debounce stable[2558] 2 for config DestinationRule/openshift-ingress/payload-processing-e2e-worker-w2-865066 and 1 more configs: 100.958821ms since last change, 200.554268ms since last push, full=true 2026-11-01T04:33:46.399598Z info ads XDS: Pushing Services:116 ConnectedEndpoints:4 Version:2026-11-01T04:33:46Z/1730 2026-11-01T04:33:46.400080Z info delta CDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:1 removed:0 size:44B 2026-11-01T04:33:46.400259Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:7 removed:0 size:1.7kB 2026-11-01T04:33:46.400298Z info delta CDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:5 removed:0 size:748B 2026-11-01T04:33:46.400306Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:44B 2026-11-01T04:33:46.400930Z info delta EDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:147 removed:0 size:35.3kB empty:0 cached:147/147 2026-11-01T04:33:46.400967Z warn no virtual service bound to gateway: openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https 2026-11-01T04:33:46.400975Z warn gateway openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https:443 listener missed network filter 2026-11-01T04:33:46.400994Z info gateway omitting listener "0.0.0.0_443" due to: must have more than 0 chains in listener "0.0.0.0_443" 2026-11-01T04:33:46.401018Z info delta LDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:0 removed:0 size:0B 2026-11-01T04:33:46.401080Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:147 removed:0 size:35.3kB empty:0 cached:147/147 2026-11-01T04:33:46.401230Z info delta EDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:147 removed:0 size:35.3kB empty:0 cached:147/147 2026-11-01T04:33:46.401344Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:147 removed:0 size:35.3kB empty:0 cached:147/147 2026-11-01T04:33:46.401440Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:7.2kB 2026-11-01T04:33:46.401554Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-11-01T04:33:46.402152Z info delta LDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:175.8kB 2026-11-01T04:33:46.402694Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:310.2kB 2026-11-01T04:33:46.403913Z info delta RDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:63.3kB cached:0/0 2026-11-01T04:33:46.405208Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:111.6kB cached:0/0 2026-11-01T04:33:53.193069Z info model Incremental push, service distinct-2-w2-865066-kserve-workload-svc.e2e-models-e2e-worker-w2-865066.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2026-11-01T04:33:53.247281Z info model Incremental push, service distinct-w2-865066-kserve-workload-svc.e2e-models-e2e-worker-w2-865066.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2026-11-01T04:33:53.348052Z info ads Push debounce stable[2559] 2 for config ServiceEntry/e2e-models-e2e-worker-w2-865066/distinct-2-w2-865066-kserve-workload-svc.e2e-models-e2e-worker-w2-865066.svc.cluster.local and 1 more configs: 100.69323ms since last change, 154.902308ms since last push, full=false 2026-11-01T04:33:53.348106Z info ads XDS: Incremental Pushing ConnectedEndpoints:4 Version:2026-11-01T04:33:46Z/1730 2026-11-01T04:33:53.355668Z info model Incremental push, service simulator-w2-865066-kserve-workload-svc.e2e-models-e2e-worker-w2-865066.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2026-11-01T04:33:53.456241Z info ads Push debounce stable[2560] 1 for config ServiceEntry/e2e-models-e2e-worker-w2-865066/simulator-w2-865066-kserve-workload-svc.e2e-models-e2e-worker-w2-865066.svc.cluster.local: 100.534474ms since last change, 100.534308ms since last push, full=false 2026-11-01T04:33:53.456312Z info ads XDS: Incremental Pushing ConnectedEndpoints:4 Version:2026-11-01T04:33:46Z/1730 2026-11-01T04:33:53.466208Z info model Incremental push, service premium-w2-865066-kserve-workload-svc.e2e-models-e2e-worker-w2-865066.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2026-11-01T04:33:53.567144Z info ads Push debounce stable[2561] 1 for config ServiceEntry/e2e-models-e2e-worker-w2-865066/premium-w2-865066-kserve-workload-svc.e2e-models-e2e-worker-w2-865066.svc.cluster.local: 100.897962ms since last change, 100.897898ms since last push, full=false 2026-11-01T04:33:53.567219Z info ads XDS: Incremental Pushing ConnectedEndpoints:4 Version:2026-11-01T04:33:46Z/1730 2026-11-01T04:33:54.421278Z info gateway reconciling gateway with controller openshift.io/gateway-controller/v1 gateway=openshift-ingress/data-science-gateway 2026-11-01T04:33:54.421313Z info gateway reconciling gateway=openshift-ingress/data-science-gateway 2026-11-01T04:33:54.697889Z info gateway gateway updated gateway=openshift-ingress/data-science-gateway 2026-11-01T04:33:54.697933Z info gateway reconciling gateway with controller openshift.io/gateway-controller/v1 gateway=openshift-ingress/maas-default-gateway 2026-11-01T04:33:54.697950Z info gateway reconciling gateway=openshift-ingress/maas-default-gateway 2026-11-01T04:33:54.977202Z info gateway gateway updated gateway=openshift-ingress/maas-default-gateway 2026-11-01T04:33:55.378805Z info ads Push Status: { "pilot_duplicate_envoy_clusters": { "kuadrant-auth-service": { "proxy": "e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress", "message": "Duplicate cluster kuadrant-auth-service found while pushing CDS" } }, "pilot_eds_no_instances": { "outbound|8000||distinct-2-w2-865066-kserve-workload-svc.e2e-models-e2e-worker-w2-865066.svc.cluster.local": {}, "outbound|8000||distinct-w2-865066-kserve-workload-svc.e2e-models-e2e-worker-w2-865066.svc.cluster.local": {}, "outbound|8000||premium-w2-865066-kserve-workload-svc.e2e-models-e2e-worker-w2-865066.svc.cluster.local": {}, "outbound|8000||simulator-w2-865066-kserve-workload-svc.e2e-models-e2e-worker-w2-865066.svc.cluster.local": {} } } 2026-11-01T04:34:00.753075Z info model Incremental push, service maas-api-e2e-worker-w2-865066.odh-ai-gateway-infra.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2026-11-01T04:34:00.802168Z error status Encountered unexpected error updating status for &{{gateway.networking.k8s.io/v1beta1/HTTPRoute d346fff9-e5ca-4d3f-980d-2b7a076b1bce maas-api-route-e2e-worker-w2-865066 odh-ai-gateway-infra cluster.local map[app.kubernetes.io/component:api app.kubernetes.io/name:maas-api app.kubernetes.io/part-of:models-as-a-service app.opendatahub.io/modelsasservice:true maas.opendatahub.io/tenant-name:e2e-worker-w2-865066 maas.opendatahub.io/tenant-namespace:ai-tenant-e2e-worker-w2-865066] map[] 27586 2026-11-01 03:54:46 +0000 UTC [{maas.opendatahub.io/v1alpha1 Config default 2a10dd41-bd58-4d90-b541-63f4534985d1 0xd164cf80287 0xd164cf80286}] 1} 0xd1647bffd88 0xd1653814888 map[]}, will try again later: httproutes.gateway.networking.k8s.io "maas-api-route-e2e-worker-w2-865066" not found 2026-11-01T04:34:00.899390Z info ads Push debounce stable[2562] 7 for config ServiceEntry/odh-ai-gateway-infra/maas-api-e2e-worker-w2-865066.odh-ai-gateway-infra.svc.cluster.local and 5 more configs: 100.355749ms since last change, 146.674295ms since last push, full=true 2026-11-01T04:34:00.914184Z info ads XDS: Pushing Services:115 ConnectedEndpoints:4 Version:2026-11-01T04:34:00Z/1731 2026-11-01T04:34:00.915561Z info delta CDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:147 removed:1 size:170.4kB cached:145/146 2026-11-01T04:34:00.915562Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:147 removed:1 size:170.4kB cached:141/146 2026-11-01T04:34:00.916219Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:153 removed:1 size:172.1kB cached:141/146 2026-11-01T04:34:00.916592Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:147 removed:0 size:35.0kB empty:1 cached:145/147 2026-11-01T04:34:00.916720Z info delta EDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:147 removed:0 size:35.0kB empty:1 cached:146/147 2026-11-01T04:34:00.916808Z warn no virtual service bound to gateway: openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https 2026-11-01T04:34:00.916820Z warn gateway openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https:443 listener missed network filter 2026-11-01T04:34:00.916835Z info gateway omitting listener "0.0.0.0_443" due to: must have more than 0 chains in listener "0.0.0.0_443" 2026-11-01T04:34:00.916856Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:147 removed:0 size:35.0kB empty:1 cached:146/147 2026-11-01T04:34:00.916861Z info delta LDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:0 removed:0 size:0B 2026-11-01T04:34:00.916930Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:7.2kB 2026-11-01T04:34:00.917077Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-11-01T04:34:00.917129Z info delta SDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:4.6kB cached:1/1 2026-11-01T04:34:00.918057Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:310.2kB 2026-11-01T04:34:00.919876Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:111.6kB cached:0/0 2026-11-01T04:34:00.920026Z info delta CDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:151 removed:1 size:171.1kB cached:0/146 2026-11-01T04:34:00.920072Z info delta SDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:4.5kB cached:1/1 2026-11-01T04:34:00.920485Z info delta EDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:147 removed:0 size:35.0kB empty:1 cached:146/147 2026-11-01T04:34:00.921452Z info delta LDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:175.5kB 2026-11-01T04:34:00.922836Z info delta RDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:60.1kB cached:0/0 2026-11-01T04:34:00.922871Z info delta SDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:4.5kB cached:1/1 2026-11-01T04:34:01.194574Z info ads Push debounce stable[2563] 3 for config DestinationRule/openshift-ingress/maas-api-backend-tls-e2e-worker-w2-865066 and 2 more configs: 100.555427ms since last change, 216.966457ms since last push, full=true 2026-11-01T04:34:01.195292Z info ads XDS: Pushing Services:115 ConnectedEndpoints:4 Version:2026-11-01T04:34:01Z/1732 2026-11-01T04:34:01.195920Z info delta CDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:2 removed:0 size:1.4kB cached:0/1 2026-11-01T04:34:01.195974Z info delta CDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:6 removed:0 size:2.1kB cached:0/1 2026-11-01T04:34:01.196016Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:2 removed:0 size:1.4kB cached:1/1 2026-11-01T04:34:01.196112Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:8 removed:0 size:3.0kB cached:0/1 2026-11-01T04:34:01.196871Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:146 removed:0 size:34.9kB empty:0 cached:146/146 2026-11-01T04:34:01.196891Z info delta EDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:146 removed:0 size:34.9kB empty:0 cached:145/146 2026-11-01T04:34:01.196922Z warn no virtual service bound to gateway: openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https 2026-11-01T04:34:01.196936Z warn gateway openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https:443 listener missed network filter 2026-11-01T04:34:01.196939Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:146 removed:0 size:34.9kB empty:0 cached:146/146 2026-11-01T04:34:01.196951Z info gateway omitting listener "0.0.0.0_443" due to: must have more than 0 chains in listener "0.0.0.0_443" 2026-11-01T04:34:01.196976Z info delta LDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:0 removed:0 size:0B 2026-11-01T04:34:01.196891Z info delta EDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:146 removed:0 size:34.9kB empty:0 cached:146/146 2026-11-01T04:34:01.197237Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:7.2kB 2026-11-01T04:34:01.197336Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-11-01T04:34:01.197627Z info delta LDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:175.4kB 2026-11-01T04:34:01.198735Z info delta RDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:60.1kB cached:0/0 2026-11-01T04:34:01.199106Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:310.2kB 2026-11-01T04:34:01.201129Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:111.6kB cached:0/0 2026-11-01T04:34:01.638699Z info ads Push debounce stable[2564] 1 for config EnvoyFilter/openshift-ingress/maas-model-access-logs-e2e-worker-w2-865066: 101.06796ms since last change, 101.067889ms since last push, full=true 2026-11-01T04:34:01.638900Z info ads XDS: Pushing Services:115 ConnectedEndpoints:4 Version:2026-11-01T04:34:01Z/1733 2026-11-01T04:34:01.641810Z info delta CDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:147 removed:0 size:170.4kB cached:146/146 2026-11-01T04:34:01.641868Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:147 removed:0 size:170.4kB cached:146/146 2026-11-01T04:34:01.642012Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:153 removed:0 size:172.1kB cached:146/146 2026-11-01T04:34:01.642687Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:146 removed:0 size:34.9kB empty:0 cached:146/146 2026-11-01T04:34:01.642695Z info delta EDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:146 removed:0 size:34.9kB empty:0 cached:146/146 2026-11-01T04:34:01.642730Z warn no virtual service bound to gateway: openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https 2026-11-01T04:34:01.642739Z warn gateway openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https:443 listener missed network filter 2026-11-01T04:34:01.642758Z info gateway omitting listener "0.0.0.0_443" due to: must have more than 0 chains in listener "0.0.0.0_443" 2026-11-01T04:34:01.642786Z info delta LDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:0 removed:0 size:0B 2026-11-01T04:34:01.642809Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:146 removed:0 size:34.9kB empty:0 cached:146/146 2026-11-01T04:34:01.642979Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:7.2kB 2026-11-01T04:34:01.643145Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-11-01T04:34:01.643866Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:310.2kB 2026-11-01T04:34:01.645159Z info delta CDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:150 removed:1 size:171.0kB cached:0/146 2026-11-01T04:34:01.645549Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:111.6kB cached:0/0 2026-11-01T04:34:01.645640Z info delta EDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:146 removed:0 size:34.9kB empty:0 cached:146/146 2026-11-01T04:34:01.646339Z info delta LDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:169.2kB 2026-11-01T04:34:01.647177Z info delta RDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:60.1kB cached:0/0 2026-11-01T04:34:02.698690Z info ads Push debounce stable[2565] 1 for config Gateway/openshift-ingress/e2e-worker-w2-865066~istio-autogenerated-k8s-gateway~https: 100.229515ms since last change, 100.229408ms since last push, full=true 2026-11-01T04:34:02.698833Z info ads XDS: Pushing Services:115 ConnectedEndpoints:4 Version:2026-11-01T04:34:02Z/1734 2026-11-01T04:34:02.699007Z warn no virtual service bound to gateway: openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https 2026-11-01T04:34:02.699023Z warn gateway openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https:443 listener missed network filter 2026-11-01T04:34:02.699061Z info gateway omitting listener "0.0.0.0_443" due to: must have more than 0 chains in listener "0.0.0.0_443" 2026-11-01T04:34:02.699122Z info delta LDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:0 removed:0 size:0B 2026-11-01T04:34:02.699223Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:7.2kB 2026-11-01T04:34:02.699329Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-11-01T04:34:02.699749Z info delta LDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:169.2kB 2026-11-01T04:34:02.700172Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:310.2kB 2026-11-01T04:34:02.700945Z info delta RDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:60.1kB cached:0/0 2026-11-01T04:34:02.702079Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:111.6kB cached:0/0 2026-11-01T04:34:07.868291Z info ads Push debounce stable[2566] 2 for config EnvoyFilter/openshift-ingress/kuadrant-ratelimiting-e2e-worker-w2-865066 and 1 more configs: 100.95739ms since last change, 103.759162ms since last push, full=true 2026-11-01T04:34:07.868537Z info ads XDS: Pushing Services:115 ConnectedEndpoints:4 Version:2026-11-01T04:34:07Z/1735 2026-11-01T04:34:07.869770Z info delta CDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:147 removed:0 size:170.4kB cached:146/146 2026-11-01T04:34:07.869777Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:147 removed:0 size:170.4kB cached:146/146 2026-11-01T04:34:07.870114Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:153 removed:0 size:172.1kB cached:146/146 2026-11-01T04:34:07.870816Z info delta EDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:146 removed:0 size:34.9kB empty:0 cached:146/146 2026-11-01T04:34:07.870854Z warn no virtual service bound to gateway: openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https 2026-11-01T04:34:07.870858Z warn gateway openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https:443 listener missed network filter 2026-11-01T04:34:07.870873Z info gateway omitting listener "0.0.0.0_443" due to: must have more than 0 chains in listener "0.0.0.0_443" 2026-11-01T04:34:07.870901Z info delta LDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:0 removed:0 size:0B 2026-11-01T04:34:07.870949Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:146 removed:0 size:34.9kB empty:0 cached:146/146 2026-11-01T04:34:07.871002Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:146 removed:0 size:34.9kB empty:0 cached:146/146 2026-11-01T04:34:07.871390Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:7.2kB 2026-11-01T04:34:07.871494Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-11-01T04:34:07.872213Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:310.2kB 2026-11-01T04:34:07.874185Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:111.6kB cached:0/0 2026-11-01T04:34:07.874274Z info delta CDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:149 removed:1 size:170.9kB cached:0/146 2026-11-01T04:34:07.874846Z info delta EDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:146 removed:0 size:34.9kB empty:0 cached:146/146 2026-11-01T04:34:07.875560Z info delta LDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:169.2kB 2026-11-01T04:34:07.876571Z info delta RDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:60.1kB cached:0/0 2026-11-01T04:34:08.353104Z info ads Push debounce stable[2567] 1 for config EnvoyFilter/openshift-ingress/kuadrant-e2e-worker-w2-865066: 100.16471ms since last change, 100.164627ms since last push, full=true 2026-11-01T04:34:08.353325Z info ads XDS: Pushing Services:115 ConnectedEndpoints:4 Version:2026-11-01T04:34:08Z/1736 2026-11-01T04:34:08.354564Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:147 removed:0 size:170.4kB cached:146/146 2026-11-01T04:34:08.354634Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:153 removed:0 size:172.1kB cached:146/146 2026-11-01T04:34:08.354747Z info delta CDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:147 removed:0 size:170.4kB cached:146/146 2026-11-01T04:34:08.354751Z info delta CDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:148 removed:1 size:170.7kB cached:132/146 2026-11-01T04:34:08.355372Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:146 removed:0 size:34.9kB empty:0 cached:146/146 2026-11-01T04:34:08.355616Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:7.2kB 2026-11-01T04:34:08.355726Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-11-01T04:34:08.356118Z info delta EDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:146 removed:0 size:34.9kB empty:0 cached:146/146 2026-11-01T04:34:08.356155Z warn no virtual service bound to gateway: openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https 2026-11-01T04:34:08.356160Z warn gateway openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https:443 listener missed network filter 2026-11-01T04:34:08.356167Z info delta EDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:146 removed:0 size:34.9kB empty:0 cached:146/146 2026-11-01T04:34:08.356176Z info gateway omitting listener "0.0.0.0_443" due to: must have more than 0 chains in listener "0.0.0.0_443" 2026-11-01T04:34:08.356199Z info delta LDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:0 removed:0 size:0B 2026-11-01T04:34:08.356359Z info delta LDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:3.9kB 2026-11-01T04:34:08.356654Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:146 removed:0 size:34.9kB empty:0 cached:146/146 2026-11-01T04:34:08.357842Z info delta RDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:60.1kB cached:0/0 2026-11-01T04:34:08.357974Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:310.2kB 2026-11-01T04:34:08.362496Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:111.6kB cached:0/0 2026-11-01T04:34:08.630356Z info ads Push debounce stable[2568] 3 for config Secret/ai-tenant-e2e-worker-w2-865066/builder-dockercfg-kckng and 2 more configs: 100.246769ms since last change, 105.025838ms since last push, full=false 2026-11-01T04:34:08.630431Z info ads XDS: Incremental Pushing ConnectedEndpoints:4 Version:2026-11-01T04:34:08Z/1736 2026-11-01T04:34:09.169515Z info ads Push debounce stable[2569] 1 for config ConfigMap/ai-tenant-e2e-worker-w2-865066/kube-root-ca.crt: 100.351948ms since last change, 100.351891ms since last push, full=false 2026-11-01T04:34:09.169564Z info ads XDS: Incremental Pushing ConnectedEndpoints:4 Version:2026-11-01T04:34:08Z/1736 2026-11-01T04:34:09.412624Z info gateway reconciling gateway with controller openshift.io/gateway-controller/v1 gateway=openshift-ingress/openshift-ai-inference 2026-11-01T04:34:09.412665Z info gateway reconciling gateway=openshift-ingress/openshift-ai-inference 2026-11-01T04:34:09.690870Z info gateway gateway updated gateway=openshift-ingress/openshift-ai-inference 2026-11-01T04:34:09.690909Z info gateway reconciling gateway with controller openshift.io/gateway-controller/v1 gateway=openshift-ingress/data-science-gateway 2026-11-01T04:34:09.690921Z info gateway reconciling gateway=openshift-ingress/data-science-gateway 2026-11-01T04:34:09.760635Z info ads Push debounce stable[2570] 1 for config Gateway/openshift-ingress/e2e-worker-w2-865066~istio-autogenerated-k8s-gateway~https: 100.14751ms since last change, 100.14744ms since last push, full=true 2026-11-01T04:34:09.760755Z info ads XDS: Pushing Services:115 ConnectedEndpoints:4 Version:2026-11-01T04:34:09Z/1737 2026-11-01T04:34:09.760891Z warn no virtual service bound to gateway: openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https 2026-11-01T04:34:09.760902Z warn gateway openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https:443 listener missed network filter 2026-11-01T04:34:09.760919Z info gateway omitting listener "0.0.0.0_443" due to: must have more than 0 chains in listener "0.0.0.0_443" 2026-11-01T04:34:09.760972Z info delta LDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:0 removed:0 size:0B 2026-11-01T04:34:09.761179Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:7.2kB 2026-11-01T04:34:09.761297Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-11-01T04:34:09.761178Z info delta LDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:3.9kB 2026-11-01T04:34:09.761760Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:310.2kB 2026-11-01T04:34:09.762306Z info delta RDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:60.1kB cached:0/0 2026-11-01T04:34:09.763715Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:111.6kB cached:0/0 2026-11-01T04:34:09.972814Z info model Incremental push, service e2e-worker-w2-865066-openshift-default.openshift-ingress.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2026-11-01T04:34:09.973070Z info gateway gateway updated gateway=openshift-ingress/data-science-gateway 2026-11-01T04:34:09.973121Z info gateway reconciling gateway with controller openshift.io/gateway-controller/v1 gateway=openshift-ingress/maas-default-gateway 2026-11-01T04:34:09.973135Z info gateway reconciling gateway=openshift-ingress/maas-default-gateway 2026-11-01T04:34:10.081857Z info ads Push debounce stable[2571] 7 for config ServiceEntry/openshift-ingress/e2e-worker-w2-865066-openshift-default.openshift-ingress.svc.cluster.local and 5 more configs: 100.342818ms since last change, 125.178147ms since last push, full=true 2026-11-01T04:34:10.082637Z info ads XDS: Pushing Services:114 ConnectedEndpoints:4 Version:2026-11-01T04:34:10Z/1738 2026-11-01T04:34:10.083840Z info delta CDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:145 removed:2 size:167.8kB cached:144/144 2026-11-01T04:34:10.084172Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:151 removed:2 size:169.4kB cached:144/144 2026-11-01T04:34:10.084258Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:145 removed:2 size:167.8kB cached:144/144 2026-11-01T04:34:10.084337Z info delta CDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:145 removed:3 size:167.8kB cached:144/144 2026-11-01T04:34:10.084498Z info delta EDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:146 removed:0 size:34.5kB empty:2 cached:144/146 2026-11-01T04:34:10.084584Z warn no virtual service bound to gateway: openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https 2026-11-01T04:34:10.084619Z warn gateway openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https:443 listener missed network filter 2026-11-01T04:34:10.084657Z info gateway omitting listener "0.0.0.0_443" due to: must have more than 0 chains in listener "0.0.0.0_443" 2026-11-01T04:34:10.084695Z info delta LDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:0 removed:0 size:0B 2026-11-01T04:34:10.085015Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:146 removed:0 size:34.5kB empty:2 cached:144/146 2026-11-01T04:34:10.085341Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:7.2kB 2026-11-01T04:34:10.085467Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-11-01T04:34:10.085564Z info delta SDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:4.6kB cached:1/1 2026-11-01T04:34:10.085341Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:146 removed:0 size:34.5kB empty:2 cached:144/146 2026-11-01T04:34:10.086448Z info delta EDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:146 removed:0 size:34.5kB empty:2 cached:144/146 2026-11-01T04:34:10.086513Z info delta LDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:0 removed:1 size:0B 2026-11-01T04:34:10.086535Z warn buildGatewayRoutes: no gateways for router e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress 2026-11-01T04:34:10.086577Z info delta RDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:98B cached:0/0 2026-11-01T04:34:10.086601Z warn ads proxy e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress attempted to access unauthorized certificates e2e-worker-w2-865066-gw-service-tls: cross namespace secret reference requires ReferenceGrant 2026-11-01T04:34:10.086604Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:310.2kB 2026-11-01T04:34:10.089539Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:111.6kB cached:0/0 2026-11-01T04:34:10.089869Z info delta SDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:4.5kB cached:1/1 2026-11-01T04:34:10.262310Z info gateway gateway updated gateway=openshift-ingress/maas-default-gateway 2026-11-01T04:34:10.473782Z info ads Push debounce stable[2572] 1 for config Secret/openshift-monitoring/prometheus-k8s: 100.108862ms since last change, 100.108808ms since last push, full=false 2026-11-01T04:34:10.473844Z info ads XDS: Incremental Pushing ConnectedEndpoints:4 Version:2026-11-01T04:34:10Z/1738 2026-11-01T04:34:10.473966Z warn ads proxy e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress attempted to access unauthorized certificates e2e-worker-w2-865066-gw-service-tls: cross namespace secret reference requires ReferenceGrant 2026-11-01T04:34:14.104934Z info ads Push debounce stable[2573] 1 for config ConfigMap/e2e-models-e2e-worker-w2-865066/kube-root-ca.crt: 100.090333ms since last change, 100.09026ms since last push, full=false 2026-11-01T04:34:14.104980Z info ads XDS: Incremental Pushing ConnectedEndpoints:4 Version:2026-11-01T04:34:10Z/1738 2026-11-01T04:34:14.105097Z warn ads proxy e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress attempted to access unauthorized certificates e2e-worker-w2-865066-gw-service-tls: cross namespace secret reference requires ReferenceGrant 2026-11-01T04:34:14.196200Z info model Incremental push, service distinct-2-w2-865066-kserve-workload-svc.e2e-models-e2e-worker-w2-865066.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2026-11-01T04:34:14.196315Z info model Incremental push, service distinct-w2-865066-kserve-workload-svc.e2e-models-e2e-worker-w2-865066.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2026-11-01T04:34:14.196371Z info model Incremental push, service premium-w2-865066-kserve-workload-svc.e2e-models-e2e-worker-w2-865066.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2026-11-01T04:34:14.196419Z info model Incremental push, service simulator-w2-865066-kserve-workload-svc.e2e-models-e2e-worker-w2-865066.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2026-11-01T04:34:14.297001Z info ads Push debounce stable[2574] 12 for config Secret/e2e-models-e2e-worker-w2-865066/deployer-dockercfg-zbzvr and 11 more configs: 100.355496ms since last change, 100.974757ms since last push, full=false 2026-11-01T04:34:14.297106Z info ads XDS: Incremental Pushing ConnectedEndpoints:4 Version:2026-11-01T04:34:10Z/1738 2026-11-01T04:34:14.297335Z warn ads proxy e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress attempted to access unauthorized certificates e2e-worker-w2-865066-gw-service-tls: cross namespace secret reference requires ReferenceGrant 2026-11-01T04:34:14.473374Z info ads Push debounce stable[2575] 4 for config DestinationRule/e2e-models-e2e-worker-w2-865066/distinct-2-w2-865066-kserve-workload-svc and 3 more configs: 100.179218ms since last change, 110.31249ms since last push, full=true 2026-11-01T04:34:14.473564Z info ads XDS: Pushing Services:114 ConnectedEndpoints:4 Version:2026-11-01T04:34:14Z/1739 2026-11-01T04:34:14.474324Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:5 removed:0 size:6.2kB cached:3/4 2026-11-01T04:34:14.474364Z info delta CDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:5 removed:0 size:6.2kB cached:1/4 2026-11-01T04:34:14.474327Z info delta CDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:5 removed:0 size:6.2kB cached:2/4 2026-11-01T04:34:14.474497Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:11 removed:0 size:7.9kB cached:0/4 2026-11-01T04:34:14.475237Z info delta EDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:146 removed:0 size:34.5kB empty:2 cached:144/146 2026-11-01T04:34:14.475256Z info delta EDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:144 removed:0 size:34.3kB empty:3 cached:141/144 2026-11-01T04:34:14.475263Z info delta EDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:144 removed:0 size:34.3kB empty:0 cached:144/144 2026-11-01T04:34:14.475269Z info delta EDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:144 removed:0 size:34.3kB empty:1 cached:143/144 2026-11-01T04:34:14.475286Z info delta LDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:0 removed:0 size:0B 2026-11-01T04:34:14.475302Z warn buildGatewayRoutes: no gateways for router e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress 2026-11-01T04:34:14.475328Z info delta RDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:98B cached:0/0 2026-11-01T04:34:14.475287Z warn no virtual service bound to gateway: openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https 2026-11-01T04:34:14.475360Z warn gateway openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https:443 listener missed network filter 2026-11-01T04:34:14.475374Z info gateway omitting listener "0.0.0.0_443" due to: must have more than 0 chains in listener "0.0.0.0_443" 2026-11-01T04:34:14.475396Z info delta LDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:0 removed:0 size:0B 2026-11-01T04:34:14.475514Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:7.2kB 2026-11-01T04:34:14.475709Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-11-01T04:34:14.476203Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:310.2kB 2026-11-01T04:34:14.478086Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:111.6kB cached:0/0 2026-11-01T04:34:15.048608Z info ads Push debounce stable[2576] 4 for config ServiceEntry/e2e-models-e2e-worker-w2-865066/simulator-w2-865066-kserve-workload-svc.e2e-models-e2e-worker-w2-865066.svc.cluster.local and 3 more configs: 100.764713ms since last change, 120.74611ms since last push, full=true 2026-11-01T04:34:15.049229Z info ads XDS: Pushing Services:110 ConnectedEndpoints:4 Version:2026-11-01T04:34:15Z/1740 2026-11-01T04:34:15.049647Z info delta CDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:4 size:44B 2026-11-01T04:34:15.049717Z info delta EDS: PUSH INC for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:0 removed:4 size:0B empty:0 cached:0/0 2026-11-01T04:34:15.049757Z info delta CDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:1 removed:4 size:44B 2026-11-01T04:34:15.049833Z info delta EDS: PUSH INC for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:0 removed:4 size:0B empty:0 cached:0/0 2026-11-01T04:34:15.049855Z warn no virtual service bound to gateway: openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https 2026-11-01T04:34:15.049861Z warn gateway openshift-ingress/openshift-ai-inference~istio-autogenerated-k8s-gateway~https:443 listener missed network filter 2026-11-01T04:34:15.049875Z info gateway omitting listener "0.0.0.0_443" due to: must have more than 0 chains in listener "0.0.0.0_443" 2026-11-01T04:34:15.049895Z info delta LDS: PUSH for node:openshift-ai-inference-openshift-ai-inference-599fb7fb95-qvpdh.openshift-ingress resources:0 removed:0 size:0B 2026-11-01T04:34:15.049903Z info delta CDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:4 size:44B 2026-11-01T04:34:15.049951Z info delta CDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:7 removed:4 size:1.7kB 2026-11-01T04:34:15.049980Z info delta LDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:7.2kB 2026-11-01T04:34:15.050017Z info delta EDS: PUSH INC for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:0 removed:4 size:0B empty:0 cached:0/0 2026-11-01T04:34:15.050019Z info delta EDS: PUSH INC for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:0 removed:4 size:0B empty:0 cached:0/0 2026-11-01T04:34:15.050074Z info delta LDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:0 removed:0 size:0B 2026-11-01T04:34:15.050091Z warn buildGatewayRoutes: no gateways for router e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress 2026-11-01T04:34:15.050115Z info delta RDS: PUSH for node:e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress resources:1 removed:0 size:98B cached:0/0 2026-11-01T04:34:15.050213Z info delta RDS: PUSH for node:data-science-gateway-data-science-gateway-class-56c7694f85sv5q4.openshift-ingress resources:1 removed:0 size:732B cached:0/0 2026-11-01T04:34:15.051169Z info delta LDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:310.2kB 2026-11-01T04:34:15.053382Z info delta RDS: PUSH for node:maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress resources:1 removed:0 size:111.6kB cached:0/0 2026-11-01T04:34:15.379103Z info ads Push Status: { "pilot_duplicate_envoy_clusters": { "kuadrant-auth-service": { "proxy": "maas-default-gateway-openshift-default-5df48fcc99-p79qx.openshift-ingress", "message": "Duplicate cluster kuadrant-auth-service found while pushing CDS" } } } 2026-11-01T04:34:20.311223Z info delta ADS: "10.132.0.69:58116" e2e-worker-w2-865066-openshift-default-6559876c99-rzp2b.openshift-ingress-118 terminated 2026-11-01T04:34:24.521204Z info gateway reconciling gateway with controller openshift.io/gateway-controller/v1 gateway=openshift-ingress/openshift-ai-inference 2026-11-01T04:34:24.521243Z info gateway reconciling gateway=openshift-ingress/openshift-ai-inference 2026-11-01T04:34:24.802139Z info gateway gateway updated gateway=openshift-ingress/openshift-ai-inference 2026-11-01T04:34:24.802181Z info gateway reconciling gateway with controller openshift.io/gateway-controller/v1 gateway=openshift-ingress/data-science-gateway 2026-11-01T04:34:24.802194Z info gateway reconciling gateway=openshift-ingress/data-science-gateway 2026-11-01T04:34:25.081750Z info gateway gateway updated gateway=openshift-ingress/data-science-gateway 2026-11-01T04:34:25.081788Z info gateway reconciling gateway with controller openshift.io/gateway-controller/v1 gateway=openshift-ingress/maas-default-gateway 2026-11-01T04:34:25.081802Z info gateway reconciling gateway=openshift-ingress/maas-default-gateway 2026-11-01T04:34:25.363299Z info gateway gateway updated gateway=openshift-ingress/maas-default-gateway