Time Namespace Component RelatedObject Reason Message

kuadrant-system

authorino-operator-7587b89b76-ntp27

Scheduled

Successfully assigned kuadrant-system/authorino-operator-7587b89b76-ntp27 to ip-10-0-131-96.ec2.internal

openshift-ingress

openshift-ai-inference-openshift-default-849cdfbf5f-m95mt

Scheduled

Successfully assigned openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-m95mt to ip-10-0-131-96.ec2.internal

kserve-ci-e2e-test

stop-feature-test-kserve-router-scheduler-5549f64b68-t6bnw

Scheduled

Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5549f64b68-t6bnw to ip-10-0-133-131.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-7659cbd84fz2m4g

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-7659cbd84fz2m4g to ip-10-0-131-96.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-6df6c7c777jj5ks

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-6df6c7c777jj5ks to ip-10-0-131-96.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1 to ip-10-0-140-48.ec2.internal

kuadrant-system

authorino-674b59b84c-wxtvv

Scheduled

Successfully assigned kuadrant-system/authorino-674b59b84c-wxtvv to ip-10-0-133-131.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6bm9hz6

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6bm9hz6 to ip-10-0-131-96.ec2.internal

kserve-ci-e2e-test

stop-feature-test-kserve-router-scheduler-5549f64b68-4j559

Scheduled

Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5549f64b68-4j559 to ip-10-0-133-131.ec2.internal

kuadrant-system

limitador-limitador-64c8f475fb-6zn8m

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-64c8f475fb-6zn8m to ip-10-0-131-96.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0 to ip-10-0-131-96.ec2.internal

openshift-ingress

istiod-openshift-gateway-798958bb55-mm958

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-798958bb55-mm958 to ip-10-0-131-96.ec2.internal

kserve-ci-e2e-test

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945dgvmgj

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-5ff945dgvmgj to ip-10-0-133-131.ec2.internal

kserve-ci-e2e-test

scheduler-ha-replicas-test-kserve-58fb968c6f-thg2j

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-58fb968c6f-thg2j to ip-10-0-133-131.ec2.internal

kserve-ci-e2e-test

router-gateway-1-openshift-default-7b9cf87876-6gstw

Scheduled

Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-6gstw to ip-10-0-131-96.ec2.internal

kserve-ci-e2e-test

router-gateway-2-openshift-default-7bb57db77b-rj2lf

Scheduled

Successfully assigned kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-rj2lf to ip-10-0-140-48.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-pd-test-kserve-prefill-7f554b9658-gq2kj

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7f554b9658-gq2kj to ip-10-0-131-96.ec2.internal

kserve-ci-e2e-test

scheduler-configmap-ref-test-kserve-router-scheduler-6cc4cdxkfv

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6cc4cdxkfv to ip-10-0-140-48.ec2.internal

kserve-ci-e2e-test

scheduler-inline-config-test-kserve-router-scheduler-84d9dbd5vs

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-84d9dbd5vs to ip-10-0-140-48.ec2.internal

kserve-ci-e2e-test

scheduler-configmap-ref-test-kserve-7867b96c5c-gzjrm

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7867b96c5c-gzjrm to ip-10-0-133-131.ec2.internal

kuadrant-system

authorino-68bd676465-488m8

Scheduled

Successfully assigned kuadrant-system/authorino-68bd676465-488m8 to ip-10-0-133-131.ec2.internal

kserve-ci-e2e-test

router-with-refs-test-kserve-router-scheduler-7cd8874bc4-m52gx

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7cd8874bc4-m52gx to ip-10-0-140-48.ec2.internal

kserve-ci-e2e-test

router-with-refs-test-kserve-67b986fbb5-tpkvf

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-test-kserve-67b986fbb5-tpkvf to ip-10-0-131-96.ec2.internal

kserve-ci-e2e-test

stop-feature-test-kserve-c5d69ddfc-qx5h4

Scheduled

Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-c5d69ddfc-qx5h4 to ip-10-0-131-96.ec2.internal

kserve-ci-e2e-test

precise-prefix-cache-test-kserve-7d6cdc99-sg8q6

Scheduled

Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7d6cdc99-sg8q6 to ip-10-0-140-48.ec2.internal

kserve-ci-e2e-test

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945drmcfw

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-5ff945drmcfw to ip-10-0-140-48.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-pd-test-kserve-7fb85f4fdc-m6pcx

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7fb85f4fdc-m6pcx to ip-10-0-131-96.ec2.internal

kserve-ci-e2e-test

router-with-refs-pd-test-kserve-router-scheduler-65bd8597-sm9wh

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65bd8597-sm9wh to ip-10-0-140-48.ec2.internal

kserve-ci-e2e-test

precise-prefix-cache-test-kserve-router-scheduler-75f8c749dvlnx

Scheduled

Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75f8c749dvlnx to ip-10-0-133-131.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-pd-test-kserve-router-scheduler-6c45f668jx

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-6c45f668jx to ip-10-0-140-48.ec2.internal

kuadrant-system

authorino-79cbc94b89-cjl4j

Scheduled

Successfully assigned kuadrant-system/authorino-79cbc94b89-cjl4j to ip-10-0-140-48.ec2.internal

kserve-ci-e2e-test

precise-prefix-cache-test-kserve-7d6cdc99-5f8v2

Scheduled

Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7d6cdc99-5f8v2 to ip-10-0-133-131.ec2.internal

kserve-ci-e2e-test

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-sche5chzm

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-sche5chzm to ip-10-0-140-48.ec2.internal

kserve-ci-e2e-test

stop-feature-test-kserve-c5d69ddfc-k9mbk

Scheduled

Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-c5d69ddfc-k9mbk to ip-10-0-131-96.ec2.internal

kserve-ci-e2e-test

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schenlh9m

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schenlh9m to ip-10-0-140-48.ec2.internal

kuadrant-system

limitador-limitador-67566c68b4-j5jj5

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-67566c68b4-j5jj5 to ip-10-0-140-48.ec2.internal

openshift-ingress

istiod-openshift-gateway-767cf5cf55-46kkm

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-767cf5cf55-46kkm to ip-10-0-131-96.ec2.internal

kserve-ci-e2e-test

router-with-refs-pd-test-kserve-655b95864c-px2qs

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-655b95864c-px2qs to ip-10-0-131-96.ec2.internal

kserve-ci-e2e-test

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches4mg2

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches4mg2 to ip-10-0-140-48.ec2.internal

kuadrant-system

limitador-operator-controller-manager-c7fb4c8d5-rbb7k

Scheduled

Successfully assigned kuadrant-system/limitador-operator-controller-manager-c7fb4c8d5-rbb7k to ip-10-0-131-96.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-test-kserve-77bdb44765-w6x4v

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-test-kserve-77bdb44765-w6x4v to ip-10-0-131-96.ec2.internal

kserve-ci-e2e-test

gw-section-name-router-with-gat-f1d92d0f-kserve-69d69bf4df6vq5q

Scheduled

Successfully assigned kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-69d69bf4df6vq5q to ip-10-0-131-96.ec2.internal

kuadrant-system

kuadrant-operator-controller-manager-57dd85bb4-pjll2

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-pjll2 to ip-10-0-133-131.ec2.internal

kserve-ci-e2e-test

scheduler-inline-config-test-kserve-c9cfc44d9-4crk8

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-inline-config-test-kserve-c9cfc44d9-4crk8 to ip-10-0-133-131.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-75cdb9b469cn8zl

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-75cdb9b469cn8zl to ip-10-0-131-96.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-test-kserve-router-scheduler-6f9b8f4fk4f8f

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6f9b8f4fk4f8f to ip-10-0-140-48.ec2.internal

kuadrant-system

kuadrant-console-plugin-6c886788f8-vnsbs

Scheduled

Successfully assigned kuadrant-system/kuadrant-console-plugin-6c886788f8-vnsbs to ip-10-0-140-48.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-b867f9f64-pt6rr

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-b867f9f64-pt6rr to ip-10-0-131-96.ec2.internal

kserve-ci-e2e-test

gw-section-name-router-with-gat-2f0a622e-kserve-c95845974-dqlvg

Scheduled

Successfully assigned kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-c95845974-dqlvg to ip-10-0-131-96.ec2.internal

kserve-ci-e2e-test

router-with-refs-pd-test-kserve-prefill-5d6b8b7978-fltrs

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5d6b8b7978-fltrs to ip-10-0-131-96.ec2.internal

kuadrant-system

dns-operator-controller-manager-844548ff4c-2xmqp

Scheduled

Successfully assigned kuadrant-system/dns-operator-controller-manager-844548ff4c-2xmqp to ip-10-0-140-48.ec2.internal

openshift-ingress

router-default-7d8488b484-56l4v

Scheduled

Successfully assigned openshift-ingress/router-default-7d8488b484-56l4v to ip-10-0-133-131.ec2.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-7d8488b484 from 0 to 1
(x9)

openshift-ingress

replicaset-controller

router-default-7d8488b484

FailedCreate

Error creating: pods "router-default-7d8488b484-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer
(x16)

openshift-ingress

replicaset-controller

router-default-7d8488b484

FailedCreate

Error creating: pods "router-default-7d8488b484-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts

openshift-ingress

replicaset-controller

router-default-7d8488b484

SuccessfulCreate

Created pod: router-default-7d8488b484-56l4v
(x5)

openshift-ingress

kubelet

router-default-7d8488b484-56l4v

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
(x5)

openshift-ingress

kubelet

router-default-7d8488b484-56l4v

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found

openshift-ingress

multus

router-default-7d8488b484-56l4v

AddedInterface

Add eth0 [10.134.0.8/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-7d8488b484-56l4v

Started

Started container router

openshift-ingress

kubelet

router-default-7d8488b484-56l4v

Created

Created container: router

openshift-ingress

kubelet

router-default-7d8488b484-56l4v

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1913f8c490e59a4bce37630afd80b56d8a165ac70c8d7c3712de13acf3d3612" already present on machine

openshift-ingress

kubelet

istiod-openshift-gateway-767cf5cf55-46kkm

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:028e10651db0d1ddb769a27c9483c6d41be6ac597f253afd9d599f395d9c82d8"

openshift-ingress

replicaset-controller

istiod-openshift-gateway-767cf5cf55

SuccessfulCreate

Created pod: istiod-openshift-gateway-767cf5cf55-46kkm

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-767cf5cf55 from 0 to 1

openshift-ingress

multus

istiod-openshift-gateway-767cf5cf55-46kkm

AddedInterface

Add eth0 [10.132.0.16/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-767cf5cf55-46kkm

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:028e10651db0d1ddb769a27c9483c6d41be6ac597f253afd9d599f395d9c82d8" in 2.672s (2.672s including waiting). Image size: 179322965 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-767cf5cf55-46kkm

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-767cf5cf55-46kkm

Started

Started container discovery

openshift-ingress

gateway_labeler_controller

openshift-ai-inference

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway openshift-ai-inference

openshift-ingress

replicaset-controller

openshift-ai-inference-openshift-default-849cdfbf5f

SuccessfulCreate

Created pod: openshift-ai-inference-openshift-default-849cdfbf5f-m95mt

openshift-ingress

service-controller

openshift-ai-inference-openshift-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

deployment-controller

openshift-ai-inference-openshift-default

ScalingReplicaSet

Scaled up replica set openshift-ai-inference-openshift-default-849cdfbf5f from 0 to 1

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-849cdfbf5f-m95mt

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:d518f3d1539f45e1253c5c9fa22062802804601d4998cd50344e476a3cc388fe"

openshift-ingress

multus

openshift-ai-inference-openshift-default-849cdfbf5f-m95mt

AddedInterface

Add eth0 [10.132.0.17/23] from ovn-kubernetes

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-849cdfbf5f-m95mt

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:d518f3d1539f45e1253c5c9fa22062802804601d4998cd50344e476a3cc388fe" in 2.578s (2.578s including waiting). Image size: 163191192 bytes.

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-849cdfbf5f-m95mt

Created

Created container: istio-proxy

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-849cdfbf5f-m95mt

Started

Started container istio-proxy

openshift-ingress

service-controller

openshift-ai-inference-openshift-default

EnsuredLoadBalancer

Ensured load balancer

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.3.0

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.3.0

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

replicaset-controller

authorino-operator-7587b89b76

SuccessfulCreate

Created pod: authorino-operator-7587b89b76-ntp27

kuadrant-system

deployment-controller

authorino-operator

ScalingReplicaSet

Scaled up replica set authorino-operator-7587b89b76 from 0 to 1

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.3.0

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.3.0

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.3.0

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.3.0

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.3.0

InstallWaiting

installing: waiting for deployment authorino-operator to become ready: deployment "authorino-operator" not available: Deployment does not have minimum availability.

kuadrant-system

kubelet

authorino-operator-7587b89b76-ntp27

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:4e581c0172549df72c6ef8978e96d771645bc36298d2ccf9240af4e71c088733"

kuadrant-system

multus

authorino-operator-7587b89b76-ntp27

AddedInterface

Add eth0 [10.132.0.18/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.3.0

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.3.0

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.3.0

InstallWaiting

installing: waiting for deployment limitador-operator-controller-manager to become ready: deployment "limitador-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

multus

limitador-operator-controller-manager-c7fb4c8d5-rbb7k

AddedInterface

Add eth0 [10.132.0.19/23] from ovn-kubernetes

kuadrant-system

authorino-operator-7587b89b76-ntp27_7429dd80-db06-455e-9434-ea438325182e

aac3a15d.authorino.kuadrant.io

LeaderElection

authorino-operator-7587b89b76-ntp27_7429dd80-db06-455e-9434-ea438325182e became leader

kuadrant-system

kubelet

authorino-operator-7587b89b76-ntp27

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:4e581c0172549df72c6ef8978e96d771645bc36298d2ccf9240af4e71c088733" in 2.561s (2.561s including waiting). Image size: 161991975 bytes.

kuadrant-system

kubelet

authorino-operator-7587b89b76-ntp27

Created

Created container: manager

kuadrant-system

kubelet

authorino-operator-7587b89b76-ntp27

Started

Started container manager

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.3.3

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.3.3

RequirementsUnknown

requirements not yet checked

kuadrant-system

deployment-controller

limitador-operator-controller-manager

ScalingReplicaSet

Scaled up replica set limitador-operator-controller-manager-c7fb4c8d5 from 0 to 1

kuadrant-system

replicaset-controller

limitador-operator-controller-manager-c7fb4c8d5

SuccessfulCreate

Created pod: limitador-operator-controller-manager-c7fb4c8d5-rbb7k

kuadrant-system

kubelet

limitador-operator-controller-manager-c7fb4c8d5-rbb7k

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:9a3ccb54bc904521277b87e1d178d8134cc6ef744eb4ec5b64177355fd5a9e66"

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.3.3

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

kubelet

limitador-operator-controller-manager-c7fb4c8d5-rbb7k

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:9a3ccb54bc904521277b87e1d178d8134cc6ef744eb4ec5b64177355fd5a9e66" in 2.671s (2.671s including waiting). Image size: 175506011 bytes.

kuadrant-system

limitador-operator-controller-manager-c7fb4c8d5-rbb7k_ec460974-0450-4c9c-8738-e5700f2360c7

3745a16e.kuadrant.io

LeaderElection

limitador-operator-controller-manager-c7fb4c8d5-rbb7k_ec460974-0450-4c9c-8738-e5700f2360c7 became leader

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-57dd85bb4 from 0 to 1

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-57dd85bb4

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-57dd85bb4-pjll2

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.3.3

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

kubelet

kuadrant-operator-controller-manager-57dd85bb4-pjll2

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:7aa6d6dbbde488260789adb94fa232a06d1e635cc2aaf86af24f8cf14a17eef8"

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.3.3

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.3.0

RequirementsUnknown

requirements not yet checked

kuadrant-system

kubelet

limitador-operator-controller-manager-c7fb4c8d5-rbb7k

Started

Started container manager

kuadrant-system

multus

kuadrant-operator-controller-manager-57dd85bb4-pjll2

AddedInterface

Add eth0 [10.134.0.26/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-operator-controller-manager-c7fb4c8d5-rbb7k

Created

Created container: manager

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.3.0

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.3.0

InstallWaiting

installing: waiting for deployment dns-operator-controller-manager to become ready: deployment "dns-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

kubelet

dns-operator-controller-manager-844548ff4c-2xmqp

Pulling

Pulling image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:b4e7ba67509320ca9ac5d63cc4add987fad05b098c4a7cd8dd91f264731177cf"

kuadrant-system

replicaset-controller

dns-operator-controller-manager-844548ff4c

SuccessfulCreate

Created pod: dns-operator-controller-manager-844548ff4c-2xmqp

kuadrant-system

deployment-controller

dns-operator-controller-manager

ScalingReplicaSet

Scaled up replica set dns-operator-controller-manager-844548ff4c from 0 to 1

kuadrant-system

multus

dns-operator-controller-manager-844548ff4c-2xmqp

AddedInterface

Add eth0 [10.133.0.43/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.3.0

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.3.0

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

kubelet

kuadrant-operator-controller-manager-57dd85bb4-pjll2

Started

Started container manager

kuadrant-system

kubelet

kuadrant-operator-controller-manager-57dd85bb4-pjll2

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:7aa6d6dbbde488260789adb94fa232a06d1e635cc2aaf86af24f8cf14a17eef8" in 3.784s (3.784s including waiting). Image size: 411782238 bytes.

kuadrant-system

kubelet

kuadrant-operator-controller-manager-57dd85bb4-pjll2

Created

Created container: manager

kuadrant-system

kuadrant-operator-controller-manager-57dd85bb4-pjll2_0e44c00c-daba-441e-b098-19dd4754117c

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-57dd85bb4-pjll2_0e44c00c-daba-441e-b098-19dd4754117c became leader

kuadrant-system

kubelet

dns-operator-controller-manager-844548ff4c-2xmqp

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:b4e7ba67509320ca9ac5d63cc4add987fad05b098c4a7cd8dd91f264731177cf" in 3.972s (3.972s including waiting). Image size: 186790172 bytes.

kuadrant-system

kubelet

dns-operator-controller-manager-844548ff4c-2xmqp

Started

Started container manager

kuadrant-system

kubelet

kuadrant-console-plugin-6c886788f8-vnsbs

FailedMount

MountVolume.SetUp failed for volume "plugin-serving-cert" : secret "plugin-serving-cert" not found

kuadrant-system

dns-operator-controller-manager-844548ff4c-2xmqp_d92b1de4-7e92-47d6-90ac-e5f5f35c0fe8

a3f98d6c.kuadrant.io

LeaderElection

dns-operator-controller-manager-844548ff4c-2xmqp_d92b1de4-7e92-47d6-90ac-e5f5f35c0fe8 became leader

kuadrant-system

kubelet

dns-operator-controller-manager-844548ff4c-2xmqp

Created

Created container: manager

kuadrant-system

replicaset-controller

kuadrant-console-plugin-6c886788f8

SuccessfulCreate

Created pod: kuadrant-console-plugin-6c886788f8-vnsbs

kuadrant-system

deployment-controller

kuadrant-console-plugin

ScalingReplicaSet

Scaled up replica set kuadrant-console-plugin-6c886788f8 from 0 to 1

kuadrant-system

kubelet

kuadrant-console-plugin-6c886788f8-vnsbs

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:41755d5e16ee5409cfe8888cf5883b262820baea4eae449fcf61f06ab1756f57"

kuadrant-system

multus

kuadrant-console-plugin-6c886788f8-vnsbs

AddedInterface

Add eth0 [10.133.0.44/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.3.0

InstallSucceeded

install strategy completed with no errors

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.3.0

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kubelet

kuadrant-console-plugin-6c886788f8-vnsbs

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:41755d5e16ee5409cfe8888cf5883b262820baea4eae449fcf61f06ab1756f57" in 4.842s (4.842s including waiting). Image size: 339541674 bytes.

kuadrant-system

kubelet

kuadrant-console-plugin-6c886788f8-vnsbs

Started

Started container kuadrant-console-plugin

kuadrant-system

kubelet

kuadrant-console-plugin-6c886788f8-vnsbs

Created

Created container: kuadrant-console-plugin

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.3.3

InstallSucceeded

install strategy completed with no errors

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.3.0

InstallSucceeded

install strategy completed with no errors

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-67566c68b4 from 0 to 1

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-674b59b84c from 0 to 1

kuadrant-system

replicaset-controller

limitador-limitador-64c8f475fb

SuccessfulCreate

Created pod: limitador-limitador-64c8f475fb-6zn8m

kuadrant-system

replicaset-controller

limitador-limitador-67566c68b4

SuccessfulCreate

Created pod: limitador-limitador-67566c68b4-j5jj5

kuadrant-system

kubelet

limitador-limitador-64c8f475fb-6zn8m

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:43bc4021bb30c7ef305e01c11e22ca5584c7e2c192b4db0a5646f04f715ee3b0"

kuadrant-system

kubelet

limitador-limitador-67566c68b4-j5jj5

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:43bc4021bb30c7ef305e01c11e22ca5584c7e2c192b4db0a5646f04f715ee3b0"

kuadrant-system

multus

limitador-limitador-64c8f475fb-6zn8m

AddedInterface

Add eth0 [10.132.0.20/23] from ovn-kubernetes

kuadrant-system

replicaset-controller

authorino-674b59b84c

SuccessfulCreate

Created pod: authorino-674b59b84c-wxtvv

kuadrant-system

multus

limitador-limitador-67566c68b4-j5jj5

AddedInterface

Add eth0 [10.133.0.45/23] from ovn-kubernetes

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-64c8f475fb from 0 to 1

kuadrant-system

multus

authorino-79cbc94b89-cjl4j

AddedInterface

Add eth0 [10.133.0.46/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-79cbc94b89-cjl4j

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c2208382e16c501e4ed58aea83f54d108567f36853cf79f7ffdf8ee54ff4ace8"

kuadrant-system

replicaset-controller

authorino-79cbc94b89

SuccessfulCreate

Created pod: authorino-79cbc94b89-cjl4j

kuadrant-system

kubelet

limitador-limitador-67566c68b4-j5jj5

Started

Started container limitador

kuadrant-system

kubelet

authorino-674b59b84c-wxtvv

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c2208382e16c501e4ed58aea83f54d108567f36853cf79f7ffdf8ee54ff4ace8"

kuadrant-system

multus

authorino-674b59b84c-wxtvv

AddedInterface

Add eth0 [10.134.0.27/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-limitador-67566c68b4-j5jj5

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:43bc4021bb30c7ef305e01c11e22ca5584c7e2c192b4db0a5646f04f715ee3b0" in 1.055s (1.055s including waiting). Image size: 249943843 bytes.

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-79cbc94b89 from 0 to 1

kuadrant-system

kubelet

limitador-limitador-67566c68b4-j5jj5

Created

Created container: limitador

kuadrant-system

kubelet

authorino-674b59b84c-wxtvv

Started

Started container authorino

kuadrant-system

kubelet

authorino-674b59b84c-wxtvv

Created

Created container: authorino

kuadrant-system

kubelet

authorino-674b59b84c-wxtvv

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c2208382e16c501e4ed58aea83f54d108567f36853cf79f7ffdf8ee54ff4ace8" in 1.508s (1.508s including waiting). Image size: 199256829 bytes.

kuadrant-system

kubelet

limitador-limitador-64c8f475fb-6zn8m

Started

Started container limitador

kuadrant-system

kubelet

limitador-limitador-64c8f475fb-6zn8m

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:43bc4021bb30c7ef305e01c11e22ca5584c7e2c192b4db0a5646f04f715ee3b0" in 3.687s (3.687s including waiting). Image size: 249943843 bytes.

kuadrant-system

replicaset-controller

authorino-674b59b84c

SuccessfulDelete

Deleted pod: authorino-674b59b84c-wxtvv

kuadrant-system

kubelet

limitador-limitador-64c8f475fb-6zn8m

Created

Created container: limitador

kuadrant-system

kubelet

authorino-79cbc94b89-cjl4j

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c2208382e16c501e4ed58aea83f54d108567f36853cf79f7ffdf8ee54ff4ace8" in 2.502s (2.503s including waiting). Image size: 199256829 bytes.

kuadrant-system

kubelet

authorino-79cbc94b89-cjl4j

Created

Created container: authorino

kuadrant-system

kubelet

authorino-79cbc94b89-cjl4j

Started

Started container authorino

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-674b59b84c from 1 to 0

kuadrant-system

kubelet

authorino-674b59b84c-wxtvv

Killing

Stopping container authorino

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled down replica set limitador-limitador-64c8f475fb from 1 to 0

kuadrant-system

kubelet

limitador-limitador-64c8f475fb-6zn8m

Killing

Stopping container limitador

kuadrant-system

replicaset-controller

limitador-limitador-64c8f475fb

SuccessfulDelete

Deleted pod: limitador-limitador-64c8f475fb-6zn8m

kuadrant-system

kubelet

authorino-68bd676465-488m8

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c2208382e16c501e4ed58aea83f54d108567f36853cf79f7ffdf8ee54ff4ace8"

kuadrant-system

replicaset-controller

authorino-68bd676465

SuccessfulCreate

Created pod: authorino-68bd676465-488m8

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-68bd676465 from 0 to 1

kuadrant-system

multus

authorino-68bd676465-488m8

AddedInterface

Add eth0 [10.134.0.28/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-68bd676465-488m8

Created

Created container: authorino

kuadrant-system

kubelet

authorino-68bd676465-488m8

Started

Started container authorino

kuadrant-system

kubelet

authorino-68bd676465-488m8

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c2208382e16c501e4ed58aea83f54d108567f36853cf79f7ffdf8ee54ff4ace8" in 918ms (918ms including waiting). Image size: 199256829 bytes.

kuadrant-system

replicaset-controller

authorino-79cbc94b89

SuccessfulDelete

Deleted pod: authorino-79cbc94b89-cjl4j

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-79cbc94b89 from 1 to 0

kuadrant-system

kubelet

authorino-79cbc94b89-cjl4j

Killing

Stopping container authorino

openshift-ingress

kubelet

istiod-openshift-gateway-798958bb55-mm958

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-798958bb55-mm958

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-798958bb55-mm958

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:028e10651db0d1ddb769a27c9483c6d41be6ac597f253afd9d599f395d9c82d8" already present on machine

openshift-ingress

multus

istiod-openshift-gateway-798958bb55-mm958

AddedInterface

Add eth0 [10.132.0.21/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

istiod-openshift-gateway-798958bb55

SuccessfulCreate

Created pod: istiod-openshift-gateway-798958bb55-mm958

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-798958bb55 from 0 to 1

openshift-ingress

kubelet

istiod-openshift-gateway-767cf5cf55-46kkm

Killing

Stopping container discovery

openshift-ingress

replicaset-controller

istiod-openshift-gateway-767cf5cf55

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-767cf5cf55-46kkm

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-767cf5cf55 from 1 to 0
(x3)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

kserve-ci-e2e-test

deployment-controller

router-gateway-1-openshift-default

ScalingReplicaSet

Scaled up replica set router-gateway-1-openshift-default-7b9cf87876 from 0 to 1

kserve-ci-e2e-test

service-controller

router-gateway-1-openshift-default

EnsuringLoadBalancer

Ensuring load balancer

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-7b9cf87876

SuccessfulCreate

Created pod: router-gateway-1-openshift-default-7b9cf87876-6gstw

kserve-ci-e2e-test

multus

router-gateway-1-openshift-default-7b9cf87876-6gstw

AddedInterface

Add eth0 [10.132.0.27/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-7b9cf87876-6gstw

Started

Started container istio-proxy

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-7b9cf87876-6gstw

Created

Created container: istio-proxy

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-7b9cf87876-6gstw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:d518f3d1539f45e1253c5c9fa22062802804601d4998cd50344e476a3cc388fe" already present on machine

kserve-ci-e2e-test

service-controller

router-gateway-1-openshift-default

EnsuredLoadBalancer

Ensured load balancer
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-inline-config-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-inline-config-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-inline-config-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-inline-config-test-kserve-route" not found

kserve-ci-e2e-test

deployment-controller

scheduler-inline-config-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-inline-config-test-kserve-c9cfc44d9 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-inline-config-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-inline-config-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-inline-config-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-inline-config-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-inline-config-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-inline-config-test-epp-role

kserve-ci-e2e-test

replicaset-controller

scheduler-inline-config-test-kserve-c9cfc44d9

SuccessfulCreate

Created pod: scheduler-inline-config-test-kserve-c9cfc44d9-4crk8
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-inline-config-test-kserve-shadow-svc

kserve-ci-e2e-test

replicaset-controller

scheduler-inline-config-test-kserve-router-scheduler-84d9d74b8c

SuccessfulCreate

Created pod: scheduler-inline-config-test-kserve-router-scheduler-84d9dbd5vs

kserve-ci-e2e-test

multus

scheduler-inline-config-test-kserve-router-scheduler-84d9dbd5vs

AddedInterface

Add eth0 [10.133.0.48/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-84d9dbd5vs

Pulling

Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341"

kserve-ci-e2e-test

deployment-controller

scheduler-inline-config-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-inline-config-test-kserve-router-scheduler-84d9d74b8c from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-inline-config-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler

kserve-ci-e2e-test

multus

scheduler-inline-config-test-kserve-c9cfc44d9-4crk8

AddedInterface

Add eth0 [10.134.0.30/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-inline-config-test-epp-service

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-c9cfc44d9-4crk8

Pulling

Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341"

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-84d9dbd5vs

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-84d9dbd5vs

Pulled

Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" in 3.152s (3.152s including waiting). Image size: 301466548 bytes.

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-c9cfc44d9-4crk8

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-c9cfc44d9-4crk8

Pulled

Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" in 3.576s (3.576s including waiting). Image size: 301466548 bytes.

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-c9cfc44d9-4crk8

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-84d9dbd5vs

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-84d9dbd5vs

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1"

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-84d9dbd5vs

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" in 1.364s (1.364s including waiting). Image size: 87241021 bytes.

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-84d9dbd5vs

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-84d9dbd5vs

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-84d9dbd5vs

Pulling

Pulling image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1"

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-c9cfc44d9-4crk8

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-c9cfc44d9-4crk8

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-c9cfc44d9-4crk8

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-c9cfc44d9-4crk8

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.165s (1.165s including waiting). Image size: 98346788 bytes.

kserve-ci-e2e-test

replicaset-controller

gw-section-name-router-with-gat-2f0a622e-kserve-c95845974

SuccessfulCreate

Created pod: gw-section-name-router-with-gat-2f0a622e-kserve-c95845974-dqlvg

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.HTTPRoute kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.DestinationRule kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-workload-svc

kserve-ci-e2e-test

deployment-controller

gw-section-name-router-with-gat-2f0a622e-kserve

ScalingReplicaSet

Scaled up replica set gw-section-name-router-with-gat-2f0a622e-kserve-c95845974 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.Secret kserve-ci-e2e-test/gw-sec0c69dceeb48768325d1a53a749e65786-kserve-self-signed-certs

kserve-ci-e2e-test

multus

gw-section-name-router-with-gat-2f0a622e-kserve-c95845974-dqlvg

AddedInterface

Add eth0 [10.132.0.28/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-c95845974-dqlvg

Pulling

Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341"

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.Deployment kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.Service kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-workload-svc

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-c95845974-dqlvg

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-c95845974-dqlvg

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-c95845974-dqlvg

Pulled

Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" in 3.476s (3.476s including waiting). Image size: 301466548 bytes.

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-c95845974-dqlvg

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-c95845974-dqlvg

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.493s (1.493s including waiting). Image size: 98346788 bytes.

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-c95845974-dqlvg

Created

Created container: main

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-c95845974-dqlvg

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-84d9dbd5vs

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-84d9dbd5vs

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" in 30.223s (30.223s including waiting). Image size: 2907020949 bytes.

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-84d9dbd5vs

Created

Created container: tokenizer

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-inline-config-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-c95845974-dqlvg

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.Deployment kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.Service kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-workload-svc

kserve-ci-e2e-test

deployment-controller

gw-section-name-router-with-gat-f1d92d0f-kserve

ScalingReplicaSet

Scaled up replica set gw-section-name-router-with-gat-f1d92d0f-kserve-69d69bf4df from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.DestinationRule kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.HTTPRoute kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-route

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-69d69bf4df6vq5q

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

multus

gw-section-name-router-with-gat-f1d92d0f-kserve-69d69bf4df6vq5q

AddedInterface

Add eth0 [10.132.0.29/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

gw-section-name-router-with-gat-f1d92d0f-kserve-69d69bf4df

SuccessfulCreate

Created pod: gw-section-name-router-with-gat-f1d92d0f-kserve-69d69bf4df6vq5q

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.Secret kserve-ci-e2e-test/gw-sec2774c263d49959f50d9eebc552e13bf9-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-69d69bf4df6vq5q

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-69d69bf4df6vq5q

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-69d69bf4df6vq5q

Killing

Stopping container storage-initializer
(x13)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-6fa1027a

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-6fa1027a

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-workload-svc

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-6df6c7c777

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-6df6c7c777jj5ks

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-6fa1027a

Created

Created v1.Secret kserve-ci-e2e-test/llmisvad71fa5348b85aebd404221bba611457-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-6fa1027a

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-6fa1027a-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-6df6c7c777 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-6fa1027a

Created

Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-6fa1027a

Created

Created v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-route

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-6df6c7c777jj5ks

Created

Created container: storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-inline-config-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-84d9dbd5vs

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-84d9dbd5vs

Killing

Stopping container tokenizer

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-c9cfc44d9-4crk8

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-6df6c7c777jj5ks

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-6df6c7c777jj5ks

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-6df6c7c777jj5ks

AddedInterface

Add eth0 [10.132.0.30/23] from ovn-kubernetes
(x14)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-configmap-ref-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-configmap-ref-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-configmap-ref-test-kserve-route" not found

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-84d9dbd5vs

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.133.0.48:9003" within 1s: context deadline exceeded

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-6df6c7c777jj5ks

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-6df6c7c777jj5ks

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-6df6c7c777jj5ks

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve

kserve-ci-e2e-test

deployment-controller

scheduler-configmap-ref-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-configmap-ref-test-kserve-7867b96c5c from 0 to 1

kserve-ci-e2e-test

replicaset-controller

scheduler-configmap-ref-test-kserve-router-scheduler-6cc4cc7f4f

SuccessfulCreate

Created pod: scheduler-configmap-ref-test-kserve-router-scheduler-6cc4cdxkfv

kserve-ci-e2e-test

multus

scheduler-configmap-ref-test-kserve-7867b96c5c-gzjrm

AddedInterface

Add eth0 [10.134.0.31/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-6cc4cdxkfv

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-6cc4cdxkfv

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-6cc4cdxkfv

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

multus

scheduler-configmap-ref-test-kserve-router-scheduler-6cc4cdxkfv

AddedInterface

Add eth0 [10.133.0.49/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

scheduler-configmap-ref-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-configmap-ref-test-kserve-router-scheduler-6cc4cc7f4f from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-configmap-ref-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-service

kserve-ci-e2e-test

replicaset-controller

scheduler-configmap-ref-test-kserve-7867b96c5c

SuccessfulCreate

Created pod: scheduler-configmap-ref-test-kserve-7867b96c5c-gzjrm

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-7867b96c5c-gzjrm

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-7867b96c5c-gzjrm

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-7867b96c5c-gzjrm

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-6cc4cdxkfv

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-6cc4cdxkfv

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-6cc4cdxkfv

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-6cc4cdxkfv

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-6cc4cdxkfv

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-6cc4cdxkfv

Started

Started container tokenizer

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-6fa1027a

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisvad71fa5348b85aebd404221bba611457-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-7867b96c5c-gzjrm

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-7867b96c5c-gzjrm

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-7867b96c5c-gzjrm

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-6df6c7c777jj5ks

Killing

Stopping container main
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-shadow-svc
(x15)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-55f7ae4a

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve

kserve-ci-e2e-test

replicaset-controller

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheduler-85954d766b

SuccessfulCreate

Created pod: llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-sche5chzm

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-epp-role

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-7659cbd84f

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-7659cbd84fz2m4g

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-7659cbd84f from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-teste2c0c7deeb42f8b9846a7a45d2caecd2-llmisvc-mode

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Created

Created v1.Secret kserve-ci-e2e-test/llmisvdde380eaa9fe1facad32d45131f9e34d-kserve-self-signed-certs

kserve-ci-e2e-test

deployment-controller

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheduler-85954d766b from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-workload-svc

kserve-ci-e2e-test

kubelet

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-sche5chzm

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

multus

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-sche5chzm

AddedInterface

Add eth0 [10.133.0.50/23] from ovn-kubernetes

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-7659cbd84fz2m4g

AddedInterface

Add eth0 [10.132.0.31/23] from ovn-kubernetes
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-7659cbd84fz2m4g

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-7659cbd84fz2m4g

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

kubelet

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-sche5chzm

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-sche5chzm

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-7659cbd84fz2m4g

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-7867b96c5c-gzjrm

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-configmap-ref-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-sche5chzm

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-sche5chzm

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-sche5chzm

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-sche5chzm

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-sche5chzm

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-sche5chzm

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-6cc4cdxkfv

Killing

Stopping container tokenizer

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-6cc4cdxkfv

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-7867b96c5c-gzjrm

Unhealthy

Readiness probe failed: Get "https://10.134.0.31:8000/health": dial tcp 10.134.0.31:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-6cc4cdxkfv

Unhealthy

Readiness probe failed: Get "http://10.133.0.49:8082/healthz": dial tcp 10.133.0.49:8082: connect: connection refused
(x14)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-ha-replicas-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-ha-replicas-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-ha-replicas-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-service

kserve-ci-e2e-test

multus

scheduler-ha-replicas-test-kserve-58fb968c6f-thg2j

AddedInterface

Add eth0 [10.134.0.32/23] from ovn-kubernetes

kserve-ci-e2e-test

multus

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945drmcfw

AddedInterface

Add eth0 [10.133.0.51/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-58fb968c6f-thg2j

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-58fb968c6f-thg2j

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-58fb968c6f-thg2j

Started

Started container storage-initializer

kserve-ci-e2e-test

replicaset-controller

scheduler-ha-replicas-test-kserve-58fb968c6f

SuccessfulCreate

Created pod: scheduler-ha-replicas-test-kserve-58fb968c6f-thg2j

kserve-ci-e2e-test

replicaset-controller

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945d6d9

SuccessfulCreate

Created pod: scheduler-ha-replicas-test-kserve-router-scheduler-5ff945drmcfw

kserve-ci-e2e-test

replicaset-controller

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945d6d9

SuccessfulCreate

Created pod: scheduler-ha-replicas-test-kserve-router-scheduler-5ff945dgvmgj

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945drmcfw

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

multus

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945dgvmgj

AddedInterface

Add eth0 [10.134.0.33/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

scheduler-ha-replicas-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-ha-replicas-test-kserve-router-scheduler-5ff945d6d9 from 0 to 2

kserve-ci-e2e-test

deployment-controller

scheduler-ha-replicas-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-ha-replicas-test-kserve-58fb968c6f from 0 to 1

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945dgvmgj

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-ha-replicas-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945dgvmgj

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1"

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945dgvmgj

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945drmcfw

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945drmcfw

Started

Started container storage-initializer
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945dgvmgj

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945dgvmgj

Pulling

Pulling image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1"

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945drmcfw

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945drmcfw

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945drmcfw

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945drmcfw

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945drmcfw

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945drmcfw

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945dgvmgj

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" in 1.057s (1.057s including waiting). Image size: 87241021 bytes.

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945dgvmgj

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945dgvmgj

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisvdde380eaa9fe1facad32d45131f9e34d-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945dgvmgj

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" in 29.2s (29.2s including waiting). Image size: 2907020949 bytes.

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945dgvmgj

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945dgvmgj

Started

Started container tokenizer

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-7659cbd84fz2m4g

Pulling

Pulling image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0"

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-7659cbd84fz2m4g

Pulled

Successfully pulled image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" in 27.744s (27.744s including waiting). Image size: 3531177328 bytes.

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-7659cbd84fz2m4g

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-7659cbd84fz2m4g

Created

Created container: main
(x10)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-7659cbd84fz2m4g

Unhealthy

Startup probe failed: Get "https://10.132.0.31:8000/health": dial tcp 10.132.0.31:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-sche5chzm

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-sche5chzm

Killing

Stopping container tokenizer

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-teste2c0c7deeb42f8b9846a7a45d2caecd2-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-7659cbd84fz2m4g

Killing

Stopping container main
(x13)

kserve-ci-e2e-test

OpenDataHubModelController

custom-route-timeout-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy custom-route-timeout-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/custom-route-timeout-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "custom-route-timeout-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Service kserve-ci-e2e-test/custom-route-timeout-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-test-kserve-router-scheduler-6f9b8f4f7d

SuccessfulCreate

Created pod: custom-route-timeout-test-kserve-router-scheduler-6f9b8f4fk4f8f

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-test-kserve-77bdb44765

SuccessfulCreate

Created pod: custom-route-timeout-test-kserve-77bdb44765-w6x4v

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set custom-route-timeout-test-kserve-router-scheduler-6f9b8f4f7d from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Role kserve-ci-e2e-test/custom-route-timeout-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Secret kserve-ci-e2e-test/custom-route-timeout-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-test-epp-sa

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-test-kserve

ScalingReplicaSet

Scaled up replica set custom-route-timeout-test-kserve-77bdb44765 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Service kserve-ci-e2e-test/custom-route-timeout-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/custom-route-timeout-test-epp-rb

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-6f9b8f4fk4f8f

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-6f9b8f4fk4f8f

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-77bdb44765-w6x4v

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/custom-route-timeout-test-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-77bdb44765-w6x4v

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-77bdb44765-w6x4v

Started

Started container storage-initializer

kserve-ci-e2e-test

multus

custom-route-timeout-test-kserve-router-scheduler-6f9b8f4fk4f8f

AddedInterface

Add eth0 [10.133.0.52/23] from ovn-kubernetes

kserve-ci-e2e-test

multus

custom-route-timeout-test-kserve-77bdb44765-w6x4v

AddedInterface

Add eth0 [10.132.0.32/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-6f9b8f4fk4f8f

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-6f9b8f4fk4f8f

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-6f9b8f4fk4f8f

Started

Started container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-6f9b8f4fk4f8f

Created

Created container: main

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-6f9b8f4fk4f8f

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-6f9b8f4fk4f8f

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-6f9b8f4fk4f8f

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-77bdb44765-w6x4v

Created

Created container: main

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-77bdb44765-w6x4v

Started

Started container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-77bdb44765-w6x4v

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Updated

Updated v1.Secret kserve-ci-e2e-test/custom-route-timeout-test-kserve-self-signed-certs
(x10)

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-77bdb44765-w6x4v

Unhealthy

Startup probe failed: Get "https://10.132.0.32:8000/health": dial tcp 10.132.0.32:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-77bdb44765-w6x4v

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-6f9b8f4fk4f8f

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-6f9b8f4fk4f8f

Killing

Stopping container tokenizer
(x4)

kserve-ci-e2e-test

gateway_labeler_controller

router-gateway-1

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway router-gateway-1

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-test-epp-auth-rb

kserve-ci-e2e-test

deployment-controller

router-with-refs-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set router-with-refs-test-kserve-router-scheduler-7cd8874bc4 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Service kserve-ci-e2e-test/router-with-refs-test-epp-service

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-67b986fbb5-tpkvf

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-67b986fbb5-tpkvf

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-67b986fbb5-tpkvf

Created

Created container: storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Deployment kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/router-with-refs-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Role kserve-ci-e2e-test/router-with-refs-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Service kserve-ci-e2e-test/router-with-refs-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Deployment kserve-ci-e2e-test/router-with-refs-test-kserve

kserve-ci-e2e-test

replicaset-controller

router-with-refs-test-kserve-67b986fbb5

SuccessfulCreate

Created pod: router-with-refs-test-kserve-67b986fbb5-tpkvf

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Secret kserve-ci-e2e-test/router-with-refs-test-kserve-self-signed-certs

kserve-ci-e2e-test

replicaset-controller

router-with-refs-test-kserve-router-scheduler-7cd8874bc4

SuccessfulCreate

Created pod: router-with-refs-test-kserve-router-scheduler-7cd8874bc4-m52gx

kserve-ci-e2e-test

multus

router-with-refs-test-kserve-67b986fbb5-tpkvf

AddedInterface

Add eth0 [10.132.0.33/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

router-with-refs-test-kserve

ScalingReplicaSet

Scaled up replica set router-with-refs-test-kserve-67b986fbb5 from 0 to 1

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-7cd8874bc4-m52gx

Started

Started container storage-initializer

kserve-ci-e2e-test

multus

router-with-refs-test-kserve-router-scheduler-7cd8874bc4-m52gx

AddedInterface

Add eth0 [10.133.0.53/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-7cd8874bc4-m52gx

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine
(x5)

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/router-with-refs-test-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-7cd8874bc4-m52gx

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-7cd8874bc4-m52gx

Started

Started container main

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-7cd8874bc4-m52gx

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-7cd8874bc4-m52gx

Created

Created container: main

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-7cd8874bc4-m52gx

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-7cd8874bc4-m52gx

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-7cd8874bc4-m52gx

Started

Started container tokenizer

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Updated

Updated v1.Secret kserve-ci-e2e-test/router-with-refs-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-67b986fbb5-tpkvf

Started

Started container main

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-67b986fbb5-tpkvf

Created

Created container: main

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-67b986fbb5-tpkvf

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine
(x68)

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945dgvmgj

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x9)

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-67b986fbb5-tpkvf

Unhealthy

Startup probe failed: Get "https://10.132.0.33:8000/health": dial tcp 10.132.0.33:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-7cd8874bc4-m52gx

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-67b986fbb5-tpkvf

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-7cd8874bc4-m52gx

Killing

Stopping container tokenizer
(x11)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-05aa9bba

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Created

Created v1.Secret kserve-ci-e2e-test/llmisv8f1a6f044e8c7a4d31a250e0c4861caf-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-b867f9f64 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-workload-svc

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6bf4576695 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test4ce8108fb3d8aff064c322bee6e173f5-llmisvc-mode

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-b867f9f64

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-b867f9f64-pt6rr

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6bf4576695

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6bm9hz6

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6bm9hz6

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-b867f9f64-pt6rr

AddedInterface

Add eth0 [10.132.0.34/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-b867f9f64-pt6rr

Pulling

Pulling image "ghcr.io/llm-d/llm-d-routing-sidecar:v0.7.1"

kserve-ci-e2e-test

replicaset-controller

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheduler-76bf6bc458

SuccessfulCreate

Created pod: llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schenlh9m
(x10)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-shadow-svc

kserve-ci-e2e-test

deployment-controller

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheduler-76bf6bc458 from 0 to 1

kserve-ci-e2e-test

multus

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schenlh9m

AddedInterface

Add eth0 [10.133.0.54/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schenlh9m

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schenlh9m

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schenlh9m

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6bm9hz6

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6bm9hz6

Created

Created container: storage-initializer

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6bm9hz6

AddedInterface

Add eth0 [10.132.0.35/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schenlh9m

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schenlh9m

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schenlh9m

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schenlh9m

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schenlh9m

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schenlh9m

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-b867f9f64-pt6rr

Started

Started container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-b867f9f64-pt6rr

Created

Created container: llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-b867f9f64-pt6rr

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-routing-sidecar:v0.7.1" in 1.035s (1.035s including waiting). Image size: 67767940 bytes.

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-b867f9f64-pt6rr

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-b867f9f64-pt6rr

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-b867f9f64-pt6rr

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-b867f9f64-pt6rr

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-b867f9f64-pt6rr

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-b867f9f64-pt6rr

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv8f1a6f044e8c7a4d31a250e0c4861caf-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6bm9hz6

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6bm9hz6

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6bm9hz6

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine
(x9)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-b867f9f64-pt6rr

Unhealthy

Startup probe failed: Get "https://10.132.0.34:8001/health": dial tcp 10.132.0.34:8001: connect: connection refused
(x8)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6bm9hz6

Unhealthy

Startup probe failed: Get "https://10.132.0.35:8000/health": dial tcp 10.132.0.35:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schenlh9m

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-b867f9f64-pt6rr

Killing

Stopping container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-b867f9f64-pt6rr

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6bm9hz6

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test4ce8108fb3d8aff064c322bee6e173f5-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schenlh9m

Killing

Stopping container tokenizer
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

custom-route-timeout-pd-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy custom-route-timeout-pd-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "custom-route-timeout-pd-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Role kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-role

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Secret kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-self-signed-certs

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-pd-test-kserve-router-scheduler-6c45f4cf5c

SuccessfulCreate

Created pod: custom-route-timeout-pd-test-kserve-router-scheduler-6c45f668jx

kserve-ci-e2e-test

multus

custom-route-timeout-pd-test-kserve-router-scheduler-6c45f668jx

AddedInterface

Add eth0 [10.133.0.55/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-rb

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Service kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-pd-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-pd-test-epp-auth-rb

kserve-ci-e2e-test

multus

custom-route-timeout-pd-test-kserve-7fb85f4fdc-m6pcx

AddedInterface

Add eth0 [10.132.0.36/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-7fb85f4fdc-m6pcx

Pulled

Container image "ghcr.io/llm-d/llm-d-routing-sidecar:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-7fb85f4fdc-m6pcx

Created

Created container: llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-7fb85f4fdc-m6pcx

Started

Started container llm-d-routing-sidecar

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-pd-test-kserve

ScalingReplicaSet

Scaled up replica set custom-route-timeout-pd-test-kserve-7fb85f4fdc from 0 to 1

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-pd-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set custom-route-timeout-pd-test-kserve-router-scheduler-6c45f4cf5c from 0 to 1

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-7fb85f4fdc-m6pcx

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-7fb85f4fdc-m6pcx

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-7fb85f4fdc-m6pcx

Started

Started container storage-initializer

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-pd-test-kserve-7fb85f4fdc

SuccessfulCreate

Created pod: custom-route-timeout-pd-test-kserve-7fb85f4fdc-m6pcx

kserve-ci-e2e-test

multus

custom-route-timeout-pd-test-kserve-prefill-7f554b9658-gq2kj

AddedInterface

Add eth0 [10.132.0.37/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-7f554b9658-gq2kj

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-7f554b9658-gq2kj

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-7f554b9658-gq2kj

Started

Started container storage-initializer

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-pd-test-kserve-prefill-7f554b9658

SuccessfulCreate

Created pod: custom-route-timeout-pd-test-kserve-prefill-7f554b9658-gq2kj

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-pd-test-kserve-prefill

ScalingReplicaSet

Scaled up replica set custom-route-timeout-pd-test-kserve-prefill-7f554b9658 from 0 to 1

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-6c45f668jx

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-6c45f668jx

Started

Started container storage-initializer
(x10)

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-6c45f668jx

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-6c45f668jx

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-6c45f668jx

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-6c45f668jx

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-6c45f668jx

Created

Created container: main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-6c45f668jx

Started

Started container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-6c45f668jx

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Updated

Updated v1.Secret kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-ha-replicas-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945dgvmgj

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945dgvmgj

Killing

Stopping container tokenizer

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945drmcfw

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-58fb968c6f-thg2j

Killing

Stopping container storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-5ff945drmcfw

Killing

Stopping container tokenizer
(x14)

kserve-ci-e2e-test

OpenDataHubModelController

precise-prefix-cache-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy precise-prefix-cache-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/precise-prefix-cache-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "precise-prefix-cache-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Service kserve-ci-e2e-test/precise-prefix-cache-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/precise-prefix-cache-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-precise-prefix-cache-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Role kserve-ci-e2e-test/precise-prefix-cache-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Deployment kserve-ci-e2e-test/precise-prefix-cache-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Secret kserve-ci-e2e-test/precise-prefix-cache-test-kserve-self-signed-certs

kserve-ci-e2e-test

deployment-controller

precise-prefix-cache-test-kserve

ScalingReplicaSet

Scaled up replica set precise-prefix-cache-test-kserve-7d6cdc99 from 0 to 2

kserve-ci-e2e-test

replicaset-controller

precise-prefix-cache-test-kserve-7d6cdc99

SuccessfulCreate

Created pod: precise-prefix-cache-test-kserve-7d6cdc99-5f8v2

kserve-ci-e2e-test

replicaset-controller

precise-prefix-cache-test-kserve-7d6cdc99

SuccessfulCreate

Created pod: precise-prefix-cache-test-kserve-7d6cdc99-sg8q6

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7d6cdc99-sg8q6

Started

Started container storage-initializer

kserve-ci-e2e-test

multus

precise-prefix-cache-test-kserve-router-scheduler-75f8c749dvlnx

AddedInterface

Add eth0 [10.134.0.35/23] from ovn-kubernetes

kserve-ci-e2e-test

multus

precise-prefix-cache-test-kserve-7d6cdc99-5f8v2

AddedInterface

Add eth0 [10.134.0.34/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7d6cdc99-5f8v2

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7d6cdc99-5f8v2

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7d6cdc99-5f8v2

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-75f8c749dvlnx

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-75f8c749dvlnx

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-75f8c749dvlnx

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

replicaset-controller

precise-prefix-cache-test-kserve-router-scheduler-75f8c74979

SuccessfulCreate

Created pod: precise-prefix-cache-test-kserve-router-scheduler-75f8c749dvlnx

kserve-ci-e2e-test

multus

precise-prefix-cache-test-kserve-7d6cdc99-sg8q6

AddedInterface

Add eth0 [10.133.0.56/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7d6cdc99-sg8q6

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7d6cdc99-sg8q6

Created

Created container: storage-initializer
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/precise-prefix-cache-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Service kserve-ci-e2e-test/precise-prefix-cache-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Deployment kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/precise-prefix-cache-test-epp-rb

kserve-ci-e2e-test

deployment-controller

precise-prefix-cache-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set precise-prefix-cache-test-kserve-router-scheduler-75f8c74979 from 0 to 1

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-75f8c749dvlnx

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-75f8c749dvlnx

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-75f8c749dvlnx

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-75f8c749dvlnx

Created

Created container: main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-75f8c749dvlnx

Started

Started container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-75f8c749dvlnx

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7d6cdc99-5f8v2

Started

Started container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7d6cdc99-5f8v2

Created

Created container: main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7d6cdc99-5f8v2

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Updated

Updated v1.Secret kserve-ci-e2e-test/precise-prefix-cache-test-kserve-self-signed-certs
(x2)

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-75f8c749dvlnx

Unhealthy

Liveness probe failed: timeout: failed to connect service "10.134.0.35:9003" within 1s: context deadline exceeded

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-7f554b9658-gq2kj

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-7f554b9658-gq2kj

Created

Created container: main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-7f554b9658-gq2kj

Started

Started container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-7fb85f4fdc-m6pcx

Started

Started container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-7fb85f4fdc-m6pcx

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-7fb85f4fdc-m6pcx

Created

Created container: main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7d6cdc99-sg8q6

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7d6cdc99-sg8q6

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.337s (1.337s including waiting). Image size: 98346788 bytes.

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7d6cdc99-sg8q6

Created

Created container: main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7d6cdc99-sg8q6

Started

Started container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-75f8c749dvlnx

Killing

Stopping container tokenizer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7d6cdc99-5f8v2

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7d6cdc99-sg8q6

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-75f8c749dvlnx

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-precise-prefix-cache-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-75f8c749dvlnx

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.134.0.35:9003" within 1s: context deadline exceeded

kserve-ci-e2e-test

LLMInferenceServiceController

conv-test-v1a1-to-v1a2

Error

Reconciliation failed: failed to combine base-configurations: failed to get LLMInferenceServiceConfig "conv-test-v1a1-to-v1a2-config" from namespaces ["kserve-ci-e2e-test", "kserve"]: LLMInferenceServiceConfig.serving.kserve.io "conv-test-v1a1-to-v1a2-config" not found
(x10)

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-7f554b9658-gq2kj

Unhealthy

Startup probe failed: Get "https://10.132.0.37:8000/health": dial tcp 10.132.0.37:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Service kserve-ci-e2e-test/stop-feature-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-stop-feature-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-c5d69ddfc-k9mbk

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

multus

stop-feature-test-kserve-c5d69ddfc-k9mbk

AddedInterface

Add eth0 [10.132.0.38/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/stop-feature-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Role kserve-ci-e2e-test/stop-feature-test-epp-role

kserve-ci-e2e-test

replicaset-controller

stop-feature-test-kserve-router-scheduler-5549f64b68

SuccessfulCreate

Created pod: stop-feature-test-kserve-router-scheduler-5549f64b68-4j559

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/stop-feature-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Service kserve-ci-e2e-test/stop-feature-test-kserve-workload-svc

kserve-ci-e2e-test

replicaset-controller

stop-feature-test-kserve-c5d69ddfc

SuccessfulCreate

Created pod: stop-feature-test-kserve-c5d69ddfc-k9mbk

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs

kserve-ci-e2e-test

deployment-controller

stop-feature-test-kserve

ScalingReplicaSet

Scaled up replica set stop-feature-test-kserve-c5d69ddfc from 0 to 1

kserve-ci-e2e-test

deployment-controller

stop-feature-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set stop-feature-test-kserve-router-scheduler-5549f64b68 from 0 to 1

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-4j559

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-c5d69ddfc-k9mbk

Started

Started container storage-initializer
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/stop-feature-test-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-c5d69ddfc-k9mbk

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-4j559

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-4j559

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

multus

stop-feature-test-kserve-router-scheduler-5549f64b68-4j559

AddedInterface

Add eth0 [10.134.0.36/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-4j559

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-4j559

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-4j559

Created

Created container: main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-4j559

Started

Started container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-4j559

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-4j559

Created

Created container: tokenizer

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Updated

Updated v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs
(x11)

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-7fb85f4fdc-m6pcx

Unhealthy

Startup probe failed: Get "https://10.132.0.36:8001/health": dial tcp 10.132.0.36:8001: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-pd-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-6c45f668jx

Killing

Stopping container tokenizer

kserve-ci-e2e-test

deployment-controller

router-gateway-2-openshift-default

ScalingReplicaSet

Scaled up replica set router-gateway-2-openshift-default-7bb57db77b from 0 to 1

kserve-ci-e2e-test

service-controller

router-gateway-2-openshift-default

EnsuringLoadBalancer

Ensuring load balancer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-7fb85f4fdc-m6pcx

Killing

Stopping container llm-d-routing-sidecar

kserve-ci-e2e-test

gateway_labeler_controller

router-gateway-2

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway router-gateway-2

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-7fb85f4fdc-m6pcx

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-7f554b9658-gq2kj

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-6c45f668jx

Killing

Stopping container main

kserve-ci-e2e-test

replicaset-controller

router-gateway-2-openshift-default-7bb57db77b

SuccessfulCreate

Created pod: router-gateway-2-openshift-default-7bb57db77b-rj2lf

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-7bb57db77b-rj2lf

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:d518f3d1539f45e1253c5c9fa22062802804601d4998cd50344e476a3cc388fe"

kserve-ci-e2e-test

multus

router-gateway-2-openshift-default-7bb57db77b-rj2lf

AddedInterface

Add eth0 [10.133.0.57/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-6c45f668jx

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.133.0.55:9003" within 1s: context deadline exceeded

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-7bb57db77b-rj2lf

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:d518f3d1539f45e1253c5c9fa22062802804601d4998cd50344e476a3cc388fe" in 2.852s (2.852s including waiting). Image size: 163191192 bytes.

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-7bb57db77b-rj2lf

Created

Created container: istio-proxy

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-7bb57db77b-rj2lf

Started

Started container istio-proxy

kserve-ci-e2e-test

service-controller

router-gateway-2-openshift-default

EnsuredLoadBalancer

Ensured load balancer
(x3)

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-7bb57db77b-rj2lf

Unhealthy

Startup probe failed: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused

kserve-ci-e2e-test

multus

router-with-refs-pd-test-kserve-prefill-5d6b8b7978-fltrs

AddedInterface

Add eth0 [10.132.0.40/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-655b95864c-px2qs

Pulled

Container image "ghcr.io/llm-d/llm-d-routing-sidecar:v0.7.1" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Secret kserve-ci-e2e-test/router-with-refs-pd-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5d6b8b7978-fltrs

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-pd-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Role kserve-ci-e2e-test/router-with-refs-pd-test-kserve-role

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/router-with-refs-pd-test-kserve-rb

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Deployment kserve-ci-e2e-test/router-with-refs-pd-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Deployment kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Service kserve-ci-e2e-test/router-with-refs-pd-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-pd-test-epp-sa

kserve-ci-e2e-test

replicaset-controller

router-with-refs-pd-test-kserve-prefill-5d6b8b7978

SuccessfulCreate

Created pod: router-with-refs-pd-test-kserve-prefill-5d6b8b7978-fltrs

kserve-ci-e2e-test

deployment-controller

router-with-refs-pd-test-kserve-prefill

ScalingReplicaSet

Scaled up replica set router-with-refs-pd-test-kserve-prefill-5d6b8b7978 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-pd-test-epp-auth-rb

kserve-ci-e2e-test

replicaset-controller

router-with-refs-pd-test-kserve-655b95864c

SuccessfulCreate

Created pod: router-with-refs-pd-test-kserve-655b95864c-px2qs

kserve-ci-e2e-test

deployment-controller

router-with-refs-pd-test-kserve

ScalingReplicaSet

Scaled up replica set router-with-refs-pd-test-kserve-655b95864c from 0 to 1

kserve-ci-e2e-test

multus

router-with-refs-pd-test-kserve-655b95864c-px2qs

AddedInterface

Add eth0 [10.132.0.39/23] from ovn-kubernetes
(x9)

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/router-with-refs-pd-test-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5d6b8b7978-fltrs

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-655b95864c-px2qs

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-655b95864c-px2qs

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-655b95864c-px2qs

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-655b95864c-px2qs

Started

Started container llm-d-routing-sidecar

kserve-ci-e2e-test

replicaset-controller

router-with-refs-pd-test-kserve-router-scheduler-65bd8597

SuccessfulCreate

Created pod: router-with-refs-pd-test-kserve-router-scheduler-65bd8597-sm9wh

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-655b95864c-px2qs

Created

Created container: llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-65bd8597-sm9wh

Started

Started container storage-initializer

kserve-ci-e2e-test

deployment-controller

router-with-refs-pd-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set router-with-refs-pd-test-kserve-router-scheduler-65bd8597 from 0 to 1

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-65bd8597-sm9wh

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-65bd8597-sm9wh

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

multus

router-with-refs-pd-test-kserve-router-scheduler-65bd8597-sm9wh

AddedInterface

Add eth0 [10.133.0.58/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5d6b8b7978-fltrs

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-65bd8597-sm9wh

Created

Created container: main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-65bd8597-sm9wh

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-65bd8597-sm9wh

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-65bd8597-sm9wh

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-65bd8597-sm9wh

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-65bd8597-sm9wh

Started

Started container main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5d6b8b7978-fltrs

Created

Created container: main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5d6b8b7978-fltrs

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5d6b8b7978-fltrs

Started

Started container main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-655b95864c-px2qs

Started

Started container main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-655b95864c-px2qs

Created

Created container: main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-655b95864c-px2qs

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Updated

Updated v1.Secret kserve-ci-e2e-test/router-with-refs-pd-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-c5d69ddfc-k9mbk

Started

Started container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-c5d69ddfc-k9mbk

Created

Created container: main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-c5d69ddfc-k9mbk

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine
(x12)

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-655b95864c-px2qs

Unhealthy

Startup probe failed: Get "https://10.132.0.39:8001/health": dial tcp 10.132.0.39:8001: connect: connection refused
(x13)

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5d6b8b7978-fltrs

Unhealthy

Startup probe failed: Get "https://10.132.0.40:8000/health": dial tcp 10.132.0.40:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-65bd8597-sm9wh

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-65bd8597-sm9wh

Killing

Stopping container tokenizer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-655b95864c-px2qs

Killing

Stopping container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-655b95864c-px2qs

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-pd-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5d6b8b7978-fltrs

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-65bd8597-sm9wh

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.133.0.58:9003" within 1s: context deadline exceeded
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-02ac88e7

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-02ac88e7

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-02ac88e7

Created

Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-02ac88e7

Created

Created v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-route

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-75cdb9b469cn8zl

Created

Created container: storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-02ac88e7

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-75cdb9b469

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-75cdb9b469cn8zl

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-75cdb9b469cn8zl

Started

Started container storage-initializer

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-75cdb9b469cn8zl

AddedInterface

Add eth0 [10.132.0.41/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-75cdb9b469cn8zl

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-75cdb9b469 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-02ac88e7

Created

Created v1.Secret kserve-ci-e2e-test/llmisv8de1d74aab16d9cabd8b5aafeb5248e8-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-75cdb9b469cn8zl

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-75cdb9b469cn8zl

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-75cdb9b469cn8zl

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-02ac88e7

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv8de1d74aab16d9cabd8b5aafeb5248e8-kserve-self-signed-certs
(x9)

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-c5d69ddfc-k9mbk

Unhealthy

Startup probe failed: Get "https://10.132.0.38:8000/health": dial tcp 10.132.0.38:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-4j559

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.ServiceAccount kserve-ci-e2e-test/stop-feature-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Service kserve-ci-e2e-test/stop-feature-test-kserve-workload-svc

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-4j559

Killing

Stopping container tokenizer

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.InferencePool kserve-ci-e2e-test/stop-feature-test-inference-pool

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Service kserve-ci-e2e-test/stop-feature-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1alpha2.InferencePool kserve-ci-e2e-test/stop-feature-test-inference-pool

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.HTTPRoute kserve-ci-e2e-test/stop-feature-test-kserve-route

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-c5d69ddfc-k9mbk

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-4j559

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.134.0.36:9003" within 1s: context deadline exceeded
(x24)

kserve-ci-e2e-test

OpenDataHubModelController

stop-feature-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy stop-feature-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/stop-feature-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "stop-feature-test-kserve-route" not found

kserve-ci-e2e-test

deployment-controller

stop-feature-test-kserve

ScalingReplicaSet

Scaled up replica set stop-feature-test-kserve-c5d69ddfc from 0 to 1

kserve-ci-e2e-test

deployment-controller

stop-feature-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set stop-feature-test-kserve-router-scheduler-5549f64b68 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

stop-feature-test-kserve-c5d69ddfc

SuccessfulCreate

Created pod: stop-feature-test-kserve-c5d69ddfc-qx5h4

kserve-ci-e2e-test

replicaset-controller

stop-feature-test-kserve-router-scheduler-5549f64b68

SuccessfulCreate

Created pod: stop-feature-test-kserve-router-scheduler-5549f64b68-t6bnw

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-c5d69ddfc-qx5h4

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-c5d69ddfc-qx5h4

Started

Started container storage-initializer

kserve-ci-e2e-test

multus

stop-feature-test-kserve-c5d69ddfc-qx5h4

AddedInterface

Add eth0 [10.132.0.42/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-c5d69ddfc-qx5h4

Created

Created container: storage-initializer

kserve-ci-e2e-test

multus

stop-feature-test-kserve-router-scheduler-5549f64b68-t6bnw

AddedInterface

Add eth0 [10.134.0.37/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-t6bnw

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-t6bnw

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-t6bnw

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-t6bnw

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-t6bnw

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-t6bnw

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-t6bnw

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-t6bnw

Created

Created container: main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-t6bnw

Started

Started container main
(x9)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-75cdb9b469cn8zl

Unhealthy

Startup probe failed: Get "https://10.132.0.41:8000/health": dial tcp 10.132.0.41:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-75cdb9b469cn8zl

Killing

Stopping container main
(x11)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-f312f5ec

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-testc9aefdf01a18c3acb55ce247710ebb71-llmisvc-mode

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-epp-sa

kserve-ci-e2e-test

replicaset-controller

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-scheduler-67866bf598

SuccessfulCreate

Created pod: llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches4mg2

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Created

Created v1.LeaderWorkerSet kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-scc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Created

Created v1.Secret kserve-ci-e2e-test/llmisv4e643bc258191ffc517a31cd1d0ddd27-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-epp-rb

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn

GroupsProgressing

Created worker statefulset for leader pod llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

kserve-ci-e2e-test

statefulset-controller

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn

SuccessfulCreate

create Pod llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0 in StatefulSet llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn successful
(x2)

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn

GroupsProgressing

Replicas are progressing, with 0 groups ready of total 1 groups

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn

GroupsProgressing

Created leader statefulset llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn

CreatingRevision

Creating revision with key 754774c98d for a newly created LeaderWorkerSet

kserve-ci-e2e-test

deployment-controller

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-scheduler-67866bf598 from 0 to 1

kserve-ci-e2e-test

statefulset-controller

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

SuccessfulCreate

create Pod llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1 in StatefulSet llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0 successful

kserve-ci-e2e-test

kubelet

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches4mg2

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine
(x8)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-shadow-svc

kserve-ci-e2e-test

multus

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches4mg2

AddedInterface

Add eth0 [10.133.0.60/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

AddedInterface

Add eth0 [10.132.0.43/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0413900e48a4ba69f75c45b06af88f792921f4669631dcf3a047c1fe5c1fc341" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches4mg2

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches4mg2

Created

Created container: storage-initializer

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1

AddedInterface

Add eth0 [10.133.0.59/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches4mg2

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches4mg2

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches4mg2

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches4mg2

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches4mg2

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches4mg2

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv4e643bc258191ffc517a31cd1d0ddd27-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-c5d69ddfc-qx5h4

Created

Created container: main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-c5d69ddfc-qx5h4

Started

Started container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-c5d69ddfc-qx5h4

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1

Pulling

Pulling image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0"
(x10)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

Unhealthy

Startup probe failed: Get "https://10.132.0.43:8000/health": dial tcp 10.132.0.43:8000: connect: connection refused

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn

AllGroupsReady

All replicas are ready, with 1 groups ready of total 1 groups

kserve-ci-e2e-test

kubelet

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches4mg2

Killing

Stopping container tokenizer

kserve-ci-e2e-test

kubelet

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches4mg2

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-testc9aefdf01a18c3acb55ce247710ebb71-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1

Pulled

Successfully pulled image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" in 41.58s (41.58s including waiting). Image size: 3531177328 bytes.

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1

Started

Started container main
(x2)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (llmisvc-model-fb-opt-125m-route-f312f5-cb7fb8cf-dockercfg-2m5pn); attempting to pull the image may not succeed.

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1

Killing

Stopping container main
(x11)

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-c5d69ddfc-qx5h4

Unhealthy

Startup probe failed: Get "https://10.132.0.42:8000/health": dial tcp 10.132.0.42:8000: connect: connection refused
(x4)

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

(combined from similar events): Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-stop-feature-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-c5d69ddfc-qx5h4

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-t6bnw

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-t6bnw

Killing

Stopping container tokenizer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-5549f64b68-t6bnw

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.134.0.37:9003" within 1s: context deadline exceeded
(x7)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1

FailedMount

MountVolume.SetUp failed for volume "tls-certs" : secret "llmisv4e643bc258191ffc517a31cd1d0ddd27-kserve-self-signed-certs" not found