Time Namespace Component RelatedObject Reason Message

kserve-ci-e2e-test

custom-route-timeout-test-kserve-router-scheduler-5c699bbd8m7bx

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5c699bbd8m7bx to ip-10-0-141-251.ec2.internal

kserve-ci-e2e-test

scheduler-ha-replicas-test-kserve-865c7d9d55-dgjrx

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-865c7d9d55-dgjrx to ip-10-0-140-71.ec2.internal

kserve-ci-e2e-test

scheduler-inline-config-test-kserve-5748858bf9-zzm67

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5748858bf9-zzm67 to ip-10-0-140-71.ec2.internal

kserve-ci-e2e-test

router-with-refs-test-kserve-fcd884cb7-x6msw

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-test-kserve-fcd884cb7-x6msw to ip-10-0-129-117.ec2.internal

kuadrant-system

kuadrant-console-plugin-6c886788f8-r4lmp

Scheduled

Successfully assigned kuadrant-system/kuadrant-console-plugin-6c886788f8-r4lmp to ip-10-0-129-117.ec2.internal

kserve-ci-e2e-test

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8m4xzm

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8m4xzm to ip-10-0-141-251.ec2.internal

kserve-ci-e2e-test

router-with-refs-pd-test-kserve-router-scheduler-55f6fbbcfxwgqm

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-55f6fbbcfxwgqm to ip-10-0-141-251.ec2.internal

kserve-ci-e2e-test

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8smmhl

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8smmhl to ip-10-0-140-71.ec2.internal

kuadrant-system

limitador-limitador-67566c68b4-2nm2m

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-67566c68b4-2nm2m to ip-10-0-141-251.ec2.internal

kserve-ci-e2e-test

router-with-refs-pd-test-kserve-prefill-8597c688bd-wbhgp

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-8597c688bd-wbhgp to ip-10-0-129-117.ec2.internal

kserve-ci-e2e-test

router-with-refs-pd-test-kserve-8c9cd8f-v6xzj

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-8c9cd8f-v6xzj to ip-10-0-129-117.ec2.internal

kserve-ci-e2e-test

router-gateway-2-openshift-default-7bb57db77b-x7rzn

Scheduled

Successfully assigned kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-x7rzn to ip-10-0-141-251.ec2.internal

kserve-ci-e2e-test

stop-feature-test-kserve-96dfd8dc5-2gs2q

Scheduled

Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-96dfd8dc5-2gs2q to ip-10-0-129-117.ec2.internal

kserve-ci-e2e-test

stop-feature-test-kserve-96dfd8dc5-2rrzb

Scheduled

Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-96dfd8dc5-2rrzb to ip-10-0-129-117.ec2.internal

kserve-ci-e2e-test

stop-feature-test-kserve-router-scheduler-6944855df4-f4h99

Scheduled

Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6944855df4-f4h99 to ip-10-0-140-71.ec2.internal

kserve-ci-e2e-test

conv-test-round-trip-kserve-56766c79cf-wgb7x

Scheduled

Successfully assigned kserve-ci-e2e-test/conv-test-round-trip-kserve-56766c79cf-wgb7x to ip-10-0-129-117.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-pd-test-kserve-c86f76cb9-dcj46

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-c86f76cb9-dcj46 to ip-10-0-129-117.ec2.internal

openshift-ingress

istiod-openshift-gateway-798958bb55-mncdp

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-798958bb55-mncdp to ip-10-0-129-117.ec2.internal

kserve-ci-e2e-test

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schedv898

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schedv898 to ip-10-0-141-251.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-684c9d6b7-t4hcw

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-684c9d6b7-t4hcw to ip-10-0-129-117.ec2.internal

kserve-ci-e2e-test

scheduler-inline-config-test-kserve-router-scheduler-95466xddxz

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-95466xddxz to ip-10-0-141-251.ec2.internal

kuadrant-system

limitador-limitador-64c8f475fb-8f8xd

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-64c8f475fb-8f8xd to ip-10-0-129-117.ec2.internal

kuadrant-system

limitador-operator-controller-manager-c7fb4c8d5-qqh27

Scheduled

Successfully assigned kuadrant-system/limitador-operator-controller-manager-c7fb4c8d5-qqh27 to ip-10-0-141-251.ec2.internal

kserve-ci-e2e-test

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegkglh

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegkglh to ip-10-0-141-251.ec2.internal

openshift-ingress

router-default-656dc888b4-j7s2n

Scheduled

Successfully assigned openshift-ingress/router-default-656dc888b4-j7s2n to ip-10-0-129-117.ec2.internal

kserve-ci-e2e-test

gw-section-name-router-with-gat-f1d92d0f-kserve-746bf7d5f-nzdcd

Scheduled

Successfully assigned kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-746bf7d5f-nzdcd to ip-10-0-129-117.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-6fdc6f5f95t7rxc

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-6fdc6f5f95t7rxc to ip-10-0-129-117.ec2.internal

kuadrant-system

kuadrant-operator-controller-manager-57dd85bb4-z2bt4

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-z2bt4 to ip-10-0-141-251.ec2.internal

kserve-ci-e2e-test

router-with-refs-test-kserve-router-scheduler-6dbd7f8f9b-tdmjc

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-6dbd7f8f9b-tdmjc to ip-10-0-141-251.ec2.internal

kserve-ci-e2e-test

scheduler-configmap-ref-test-kserve-router-scheduler-579b56x6tg

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-579b56x6tg to ip-10-0-141-251.ec2.internal

kserve-ci-e2e-test

gw-section-name-router-with-gat-2f0a622e-kserve-75f496cc6428bk4

Scheduled

Successfully assigned kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-75f496cc6428bk4 to ip-10-0-141-251.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-pd-test-kserve-prefill-75d85d5cb8-crdgj

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-75d85d5cb8-crdgj to ip-10-0-129-117.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-f8dd46d4b-f8l57

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-f8dd46d4b-f8l57 to ip-10-0-129-117.ec2.internal

kuadrant-system

dns-operator-controller-manager-844548ff4c-k8v9v

Scheduled

Successfully assigned kuadrant-system/dns-operator-controller-manager-844548ff4c-k8v9v to ip-10-0-141-251.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6fcl8vk

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6fcl8vk to ip-10-0-129-117.ec2.internal

kuadrant-system

authorino-operator-7587b89b76-r95s7

Scheduled

Successfully assigned kuadrant-system/authorino-operator-7587b89b76-r95s7 to ip-10-0-140-71.ec2.internal

kuadrant-system

authorino-79cbc94b89-gsf4x

Scheduled

Successfully assigned kuadrant-system/authorino-79cbc94b89-gsf4x to ip-10-0-140-71.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-5f4978c9bcksn8t

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-5f4978c9bcksn8t to ip-10-0-129-117.ec2.internal

kserve-ci-e2e-test

router-gateway-1-openshift-default-7b9cf87876-d6vqg

Scheduled

Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-d6vqg to ip-10-0-141-251.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-test-kserve-6bb559dc87-6tns4

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-test-kserve-6bb559dc87-6tns4 to ip-10-0-129-117.ec2.internal

kserve-ci-e2e-test

precise-prefix-cache-test-kserve-router-scheduler-7dbf97ff6l2d7

Scheduled

Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-7dbf97ff6l2d7 to ip-10-0-140-71.ec2.internal

kserve-ci-e2e-test

precise-prefix-cache-test-kserve-7cbfd5474f-smcw5

Scheduled

Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7cbfd5474f-smcw5 to ip-10-0-140-71.ec2.internal

openshift-ingress

openshift-ai-inference-openshift-default-84b84d7ccf-g66f7

Scheduled

Successfully assigned openshift-ingress/openshift-ai-inference-openshift-default-84b84d7ccf-g66f7 to ip-10-0-141-251.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-pd-test-kserve-router-scheduler-5cb7f48br2

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5cb7f48br2 to ip-10-0-141-251.ec2.internal

kserve-ci-e2e-test

stop-feature-test-kserve-router-scheduler-6944855df4-q7mvb

Scheduled

Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6944855df4-q7mvb to ip-10-0-140-71.ec2.internal

kuadrant-system

authorino-674b59b84c-kv47x

Scheduled

Successfully assigned kuadrant-system/authorino-674b59b84c-kv47x to ip-10-0-129-117.ec2.internal

kserve-ci-e2e-test

precise-prefix-cache-test-kserve-7cbfd5474f-6xw9c

Scheduled

Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7cbfd5474f-6xw9c to ip-10-0-129-117.ec2.internal

openshift-ingress

istiod-openshift-gateway-767cf5cf55-zzv8x

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-767cf5cf55-zzv8x to ip-10-0-141-251.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0 to ip-10-0-129-117.ec2.internal

kuadrant-system

authorino-68bd676465-4dv2w

Scheduled

Successfully assigned kuadrant-system/authorino-68bd676465-4dv2w to ip-10-0-140-71.ec2.internal

kserve-ci-e2e-test

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schedx5gc

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schedx5gc to ip-10-0-141-251.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1 to ip-10-0-141-251.ec2.internal

kserve-ci-e2e-test

scheduler-configmap-ref-test-kserve-59c9cf8475-gjmqb

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-59c9cf8475-gjmqb to ip-10-0-140-71.ec2.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-656dc888b4 from 0 to 1
(x13)

openshift-ingress

replicaset-controller

router-default-656dc888b4

FailedCreate

Error creating: pods "router-default-656dc888b4-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer
(x16)

openshift-ingress

replicaset-controller

router-default-656dc888b4

FailedCreate

Error creating: pods "router-default-656dc888b4-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts

openshift-ingress

replicaset-controller

router-default-656dc888b4

SuccessfulCreate

Created pod: router-default-656dc888b4-j7s2n
(x5)

openshift-ingress

kubelet

router-default-656dc888b4-j7s2n

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x6)

openshift-ingress

kubelet

router-default-656dc888b4-j7s2n

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt

openshift-ingress

kubelet

router-default-656dc888b4-j7s2n

Created

Created container: router

openshift-ingress

kubelet

router-default-656dc888b4-j7s2n

Started

Started container router

openshift-ingress

multus

router-default-656dc888b4-j7s2n

AddedInterface

Add eth0 [10.133.0.10/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-656dc888b4-j7s2n

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1913f8c490e59a4bce37630afd80b56d8a165ac70c8d7c3712de13acf3d3612" already present on machine

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-767cf5cf55 from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-767cf5cf55

SuccessfulCreate

Created pod: istiod-openshift-gateway-767cf5cf55-zzv8x

openshift-ingress

multus

istiod-openshift-gateway-767cf5cf55-zzv8x

AddedInterface

Add eth0 [10.132.0.38/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-767cf5cf55-zzv8x

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:028e10651db0d1ddb769a27c9483c6d41be6ac597f253afd9d599f395d9c82d8"

openshift-ingress

kubelet

istiod-openshift-gateway-767cf5cf55-zzv8x

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:028e10651db0d1ddb769a27c9483c6d41be6ac597f253afd9d599f395d9c82d8" in 5.636s (5.636s including waiting). Image size: 179322965 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-767cf5cf55-zzv8x

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-767cf5cf55-zzv8x

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-767cf5cf55-zzv8x

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-767cf5cf55-zzv8x

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

replicaset-controller

openshift-ai-inference-openshift-default-84b84d7ccf

SuccessfulCreate

Created pod: openshift-ai-inference-openshift-default-84b84d7ccf-g66f7

openshift-ingress

deployment-controller

openshift-ai-inference-openshift-default

ScalingReplicaSet

Scaled up replica set openshift-ai-inference-openshift-default-84b84d7ccf from 0 to 1

openshift-ingress

service-controller

openshift-ai-inference-openshift-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

gateway_labeler_controller

openshift-ai-inference

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway openshift-ai-inference

openshift-ingress

multus

openshift-ai-inference-openshift-default-84b84d7ccf-g66f7

AddedInterface

Add eth0 [10.132.0.40/23] from ovn-kubernetes

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-84b84d7ccf-g66f7

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:d518f3d1539f45e1253c5c9fa22062802804601d4998cd50344e476a3cc388fe"

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-84b84d7ccf-g66f7

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:d518f3d1539f45e1253c5c9fa22062802804601d4998cd50344e476a3cc388fe" in 2.928s (2.928s including waiting). Image size: 163191192 bytes.

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-84b84d7ccf-g66f7

Created

Created container: istio-proxy

openshift-ingress

service-controller

openshift-ai-inference-openshift-default

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-84b84d7ccf-g66f7

Started

Started container istio-proxy

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.3.3

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.3.3

RequirementsNotMet

one or more requirements couldn't be found

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.3.3

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.3.3

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.3.3

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.3.0

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.3.0

RequirementsUnknown

requirements not yet checked

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-57dd85bb4 from 0 to 1

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-57dd85bb4

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-57dd85bb4-z2bt4

kuadrant-system

multus

kuadrant-operator-controller-manager-57dd85bb4-z2bt4

AddedInterface

Add eth0 [10.132.0.45/23] from ovn-kubernetes

kuadrant-system

kubelet

kuadrant-operator-controller-manager-57dd85bb4-z2bt4

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:7aa6d6dbbde488260789adb94fa232a06d1e635cc2aaf86af24f8cf14a17eef8"

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.3.0

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

replicaset-controller

authorino-operator-7587b89b76

SuccessfulCreate

Created pod: authorino-operator-7587b89b76-r95s7

kuadrant-system

deployment-controller

authorino-operator

ScalingReplicaSet

Scaled up replica set authorino-operator-7587b89b76 from 0 to 1

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.3.0

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.3.0

AllRequirementsMet

all requirements found, attempting install
(x2)

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.3.0

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

kubelet

authorino-operator-7587b89b76-r95s7

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:4e581c0172549df72c6ef8978e96d771645bc36298d2ccf9240af4e71c088733"

kuadrant-system

multus

authorino-operator-7587b89b76-r95s7

AddedInterface

Add eth0 [10.134.0.15/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.3.0

InstallWaiting

installing: waiting for deployment authorino-operator to become ready: deployment "authorino-operator" not available: Deployment does not have minimum availability.

kuadrant-system

kubelet

authorino-operator-7587b89b76-r95s7

Started

Started container manager

kuadrant-system

kubelet

kuadrant-operator-controller-manager-57dd85bb4-z2bt4

Created

Created container: manager

kuadrant-system

replicaset-controller

limitador-operator-controller-manager-c7fb4c8d5

SuccessfulCreate

Created pod: limitador-operator-controller-manager-c7fb4c8d5-qqh27

kuadrant-system

kuadrant-operator-controller-manager-57dd85bb4-z2bt4_6e4f7437-3fe8-4094-9688-56d9786d45fe

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-57dd85bb4-z2bt4_6e4f7437-3fe8-4094-9688-56d9786d45fe became leader

kuadrant-system

kubelet

authorino-operator-7587b89b76-r95s7

Created

Created container: manager

kuadrant-system

kubelet

authorino-operator-7587b89b76-r95s7

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:4e581c0172549df72c6ef8978e96d771645bc36298d2ccf9240af4e71c088733" in 2.447s (2.447s including waiting). Image size: 161991975 bytes.

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.3.0

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

authorino-operator-7587b89b76-r95s7_5a786e3e-0678-42b7-a7d1-f28618aa26a0

aac3a15d.authorino.kuadrant.io

LeaderElection

authorino-operator-7587b89b76-r95s7_5a786e3e-0678-42b7-a7d1-f28618aa26a0 became leader

kuadrant-system

kubelet

kuadrant-operator-controller-manager-57dd85bb4-z2bt4

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:7aa6d6dbbde488260789adb94fa232a06d1e635cc2aaf86af24f8cf14a17eef8" in 4.59s (4.59s including waiting). Image size: 411782238 bytes.

kuadrant-system

deployment-controller

limitador-operator-controller-manager

ScalingReplicaSet

Scaled up replica set limitador-operator-controller-manager-c7fb4c8d5 from 0 to 1

kuadrant-system

kubelet

kuadrant-operator-controller-manager-57dd85bb4-z2bt4

Started

Started container manager

kuadrant-system

kubelet

limitador-operator-controller-manager-c7fb4c8d5-qqh27

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:9a3ccb54bc904521277b87e1d178d8134cc6ef744eb4ec5b64177355fd5a9e66"

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.3.0

InstallWaiting

installing: waiting for deployment limitador-operator-controller-manager to become ready: deployment "limitador-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.3.0

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.3.0

RequirementsNotMet

one or more requirements couldn't be found
(x2)

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.3.0

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

multus

limitador-operator-controller-manager-c7fb4c8d5-qqh27

AddedInterface

Add eth0 [10.132.0.46/23] from ovn-kubernetes

kuadrant-system

deployment-controller

kuadrant-console-plugin

ScalingReplicaSet

Scaled up replica set kuadrant-console-plugin-6c886788f8 from 0 to 1

kuadrant-system

replicaset-controller

kuadrant-console-plugin-6c886788f8

SuccessfulCreate

Created pod: kuadrant-console-plugin-6c886788f8-r4lmp

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.3.0

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

replicaset-controller

dns-operator-controller-manager-844548ff4c

SuccessfulCreate

Created pod: dns-operator-controller-manager-844548ff4c-k8v9v

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.3.0

InstallWaiting

installing: waiting for deployment dns-operator-controller-manager to become ready: deployment "dns-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.3.0

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

deployment-controller

dns-operator-controller-manager

ScalingReplicaSet

Scaled up replica set dns-operator-controller-manager-844548ff4c from 0 to 1
(x2)

kuadrant-system

kubelet

kuadrant-console-plugin-6c886788f8-r4lmp

FailedMount

MountVolume.SetUp failed for volume "plugin-serving-cert" : secret "plugin-serving-cert" not found

kuadrant-system

multus

kuadrant-console-plugin-6c886788f8-r4lmp

AddedInterface

Add eth0 [10.133.0.27/23] from ovn-kubernetes

kuadrant-system

kubelet

kuadrant-console-plugin-6c886788f8-r4lmp

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:41755d5e16ee5409cfe8888cf5883b262820baea4eae449fcf61f06ab1756f57"

kuadrant-system

kubelet

limitador-operator-controller-manager-c7fb4c8d5-qqh27

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:9a3ccb54bc904521277b87e1d178d8134cc6ef744eb4ec5b64177355fd5a9e66" in 2.748s (2.748s including waiting). Image size: 175506011 bytes.

kuadrant-system

multus

dns-operator-controller-manager-844548ff4c-k8v9v

AddedInterface

Add eth0 [10.132.0.47/23] from ovn-kubernetes

kuadrant-system

limitador-operator-controller-manager-c7fb4c8d5-qqh27_62f18017-ce8a-46fa-a018-b70715766731

3745a16e.kuadrant.io

LeaderElection

limitador-operator-controller-manager-c7fb4c8d5-qqh27_62f18017-ce8a-46fa-a018-b70715766731 became leader

kuadrant-system

kubelet

dns-operator-controller-manager-844548ff4c-k8v9v

Pulling

Pulling image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:b4e7ba67509320ca9ac5d63cc4add987fad05b098c4a7cd8dd91f264731177cf"

kuadrant-system

kubelet

limitador-operator-controller-manager-c7fb4c8d5-qqh27

Created

Created container: manager

kuadrant-system

kubelet

limitador-operator-controller-manager-c7fb4c8d5-qqh27

Started

Started container manager

kuadrant-system

kubelet

dns-operator-controller-manager-844548ff4c-k8v9v

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:b4e7ba67509320ca9ac5d63cc4add987fad05b098c4a7cd8dd91f264731177cf" in 2.652s (2.652s including waiting). Image size: 186790172 bytes.

kuadrant-system

kubelet

dns-operator-controller-manager-844548ff4c-k8v9v

Created

Created container: manager

kuadrant-system

kubelet

dns-operator-controller-manager-844548ff4c-k8v9v

Started

Started container manager

kuadrant-system

dns-operator-controller-manager-844548ff4c-k8v9v_27ee12a6-76ee-47a7-b5f7-91d63019ab30

a3f98d6c.kuadrant.io

LeaderElection

dns-operator-controller-manager-844548ff4c-k8v9v_27ee12a6-76ee-47a7-b5f7-91d63019ab30 became leader

kuadrant-system

kubelet

kuadrant-console-plugin-6c886788f8-r4lmp

Started

Started container kuadrant-console-plugin

kuadrant-system

kubelet

kuadrant-console-plugin-6c886788f8-r4lmp

Created

Created container: kuadrant-console-plugin

kuadrant-system

kubelet

kuadrant-console-plugin-6c886788f8-r4lmp

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:41755d5e16ee5409cfe8888cf5883b262820baea4eae449fcf61f06ab1756f57" in 5.188s (5.188s including waiting). Image size: 339541674 bytes.

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.3.3

InstallSucceeded

install strategy completed with no errors

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.3.0

InstallSucceeded

install strategy completed with no errors

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.3.0

InstallSucceeded

install strategy completed with no errors

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.3.0

InstallSucceeded

install strategy completed with no errors

kuadrant-system

replicaset-controller

limitador-limitador-64c8f475fb

SuccessfulCreate

Created pod: limitador-limitador-64c8f475fb-8f8xd

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-64c8f475fb from 0 to 1

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-674b59b84c from 0 to 1

kuadrant-system

kubelet

limitador-limitador-67566c68b4-2nm2m

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:43bc4021bb30c7ef305e01c11e22ca5584c7e2c192b4db0a5646f04f715ee3b0"

kuadrant-system

replicaset-controller

limitador-limitador-67566c68b4

SuccessfulCreate

Created pod: limitador-limitador-67566c68b4-2nm2m

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-79cbc94b89 from 0 to 1

kuadrant-system

multus

limitador-limitador-67566c68b4-2nm2m

AddedInterface

Add eth0 [10.132.0.48/23] from ovn-kubernetes

kuadrant-system

replicaset-controller

authorino-79cbc94b89

SuccessfulCreate

Created pod: authorino-79cbc94b89-gsf4x

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-67566c68b4 from 0 to 1

kuadrant-system

multus

limitador-limitador-64c8f475fb-8f8xd

AddedInterface

Add eth0 [10.133.0.28/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-limitador-64c8f475fb-8f8xd

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:43bc4021bb30c7ef305e01c11e22ca5584c7e2c192b4db0a5646f04f715ee3b0"

kuadrant-system

replicaset-controller

authorino-674b59b84c

SuccessfulCreate

Created pod: authorino-674b59b84c-kv47x

kuadrant-system

kubelet

limitador-limitador-64c8f475fb-8f8xd

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:43bc4021bb30c7ef305e01c11e22ca5584c7e2c192b4db0a5646f04f715ee3b0" in 1.18s (1.18s including waiting). Image size: 249943843 bytes.

kuadrant-system

kubelet

authorino-79cbc94b89-gsf4x

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c2208382e16c501e4ed58aea83f54d108567f36853cf79f7ffdf8ee54ff4ace8"

kuadrant-system

kubelet

authorino-674b59b84c-kv47x

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c2208382e16c501e4ed58aea83f54d108567f36853cf79f7ffdf8ee54ff4ace8"

kuadrant-system

kubelet

limitador-limitador-64c8f475fb-8f8xd

Created

Created container: limitador

kuadrant-system

multus

authorino-79cbc94b89-gsf4x

AddedInterface

Add eth0 [10.134.0.16/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-limitador-64c8f475fb-8f8xd

Started

Started container limitador

kuadrant-system

multus

authorino-674b59b84c-kv47x

AddedInterface

Add eth0 [10.133.0.29/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-674b59b84c-kv47x

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c2208382e16c501e4ed58aea83f54d108567f36853cf79f7ffdf8ee54ff4ace8" in 2.482s (2.482s including waiting). Image size: 199256829 bytes.

kuadrant-system

kubelet

authorino-674b59b84c-kv47x

Started

Started container authorino

kuadrant-system

kubelet

authorino-79cbc94b89-gsf4x

Started

Started container authorino

kuadrant-system

kubelet

authorino-674b59b84c-kv47x

Created

Created container: authorino

kuadrant-system

replicaset-controller

authorino-674b59b84c

SuccessfulDelete

Deleted pod: authorino-674b59b84c-kv47x

kuadrant-system

kubelet

authorino-79cbc94b89-gsf4x

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c2208382e16c501e4ed58aea83f54d108567f36853cf79f7ffdf8ee54ff4ace8" in 2.222s (2.222s including waiting). Image size: 199256829 bytes.

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-674b59b84c from 1 to 0

kuadrant-system

kubelet

authorino-79cbc94b89-gsf4x

Created

Created container: authorino

kuadrant-system

kubelet

limitador-limitador-67566c68b4-2nm2m

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:43bc4021bb30c7ef305e01c11e22ca5584c7e2c192b4db0a5646f04f715ee3b0" in 3.984s (3.985s including waiting). Image size: 249943843 bytes.

kuadrant-system

kubelet

limitador-limitador-67566c68b4-2nm2m

Created

Created container: limitador

kuadrant-system

kubelet

limitador-limitador-67566c68b4-2nm2m

Started

Started container limitador

kuadrant-system

kubelet

authorino-674b59b84c-kv47x

Killing

Stopping container authorino

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled down replica set limitador-limitador-64c8f475fb from 1 to 0

kuadrant-system

kubelet

limitador-limitador-64c8f475fb-8f8xd

Killing

Stopping container limitador

kuadrant-system

replicaset-controller

limitador-limitador-64c8f475fb

SuccessfulDelete

Deleted pod: limitador-limitador-64c8f475fb-8f8xd

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-68bd676465 from 0 to 1

kuadrant-system

replicaset-controller

authorino-68bd676465

SuccessfulCreate

Created pod: authorino-68bd676465-4dv2w

kuadrant-system

multus

authorino-68bd676465-4dv2w

AddedInterface

Add eth0 [10.134.0.17/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-68bd676465-4dv2w

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c2208382e16c501e4ed58aea83f54d108567f36853cf79f7ffdf8ee54ff4ace8"

kuadrant-system

kubelet

authorino-68bd676465-4dv2w

Created

Created container: authorino

kuadrant-system

kubelet

authorino-68bd676465-4dv2w

Started

Started container authorino

kuadrant-system

kubelet

authorino-79cbc94b89-gsf4x

Killing

Stopping container authorino

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-79cbc94b89 from 1 to 0

kuadrant-system

kubelet

authorino-68bd676465-4dv2w

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c2208382e16c501e4ed58aea83f54d108567f36853cf79f7ffdf8ee54ff4ace8" in 501ms (501ms including waiting). Image size: 199256829 bytes.

kuadrant-system

replicaset-controller

authorino-79cbc94b89

SuccessfulDelete

Deleted pod: authorino-79cbc94b89-gsf4x

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-798958bb55 from 0 to 1

openshift-ingress

kubelet

istiod-openshift-gateway-798958bb55-mncdp

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:028e10651db0d1ddb769a27c9483c6d41be6ac597f253afd9d599f395d9c82d8"

openshift-ingress

multus

istiod-openshift-gateway-798958bb55-mncdp

AddedInterface

Add eth0 [10.133.0.30/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

istiod-openshift-gateway-798958bb55

SuccessfulCreate

Created pod: istiod-openshift-gateway-798958bb55-mncdp

openshift-ingress

kubelet

istiod-openshift-gateway-798958bb55-mncdp

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-798958bb55-mncdp

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-798958bb55-mncdp

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:028e10651db0d1ddb769a27c9483c6d41be6ac597f253afd9d599f395d9c82d8" in 3.523s (3.523s including waiting). Image size: 179322965 bytes.

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-767cf5cf55 from 1 to 0

openshift-ingress

replicaset-controller

istiod-openshift-gateway-767cf5cf55

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-767cf5cf55-zzv8x

openshift-ingress

kubelet

istiod-openshift-gateway-767cf5cf55-zzv8x

Killing

Stopping container discovery
(x4)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-7b9cf87876

SuccessfulCreate

Created pod: router-gateway-1-openshift-default-7b9cf87876-d6vqg

kserve-ci-e2e-test

deployment-controller

router-gateway-1-openshift-default

ScalingReplicaSet

Scaled up replica set router-gateway-1-openshift-default-7b9cf87876 from 0 to 1

kserve-ci-e2e-test

service-controller

router-gateway-1-openshift-default

EnsuringLoadBalancer

Ensuring load balancer

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-7b9cf87876-d6vqg

Started

Started container istio-proxy

kserve-ci-e2e-test

multus

router-gateway-1-openshift-default-7b9cf87876-d6vqg

AddedInterface

Add eth0 [10.132.0.56/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-7b9cf87876-d6vqg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:d518f3d1539f45e1253c5c9fa22062802804601d4998cd50344e476a3cc388fe" already present on machine

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-7b9cf87876-d6vqg

Created

Created container: istio-proxy

kserve-ci-e2e-test

service-controller

router-gateway-1-openshift-default

EnsuredLoadBalancer

Ensured load balancer

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.Secret kserve-ci-e2e-test/gw-sec0c69dceeb48768325d1a53a749e65786-kserve-self-signed-certs

kserve-ci-e2e-test

replicaset-controller

gw-section-name-router-with-gat-2f0a622e-kserve-75f496cc64

SuccessfulCreate

Created pod: gw-section-name-router-with-gat-2f0a622e-kserve-75f496cc6428bk4

kserve-ci-e2e-test

multus

gw-section-name-router-with-gat-2f0a622e-kserve-75f496cc6428bk4

AddedInterface

Add eth0 [10.132.0.57/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-75f496cc6428bk4

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-75f496cc6428bk4

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-75f496cc6428bk4

Started

Started container storage-initializer

kserve-ci-e2e-test

deployment-controller

gw-section-name-router-with-gat-2f0a622e-kserve

ScalingReplicaSet

Scaled up replica set gw-section-name-router-with-gat-2f0a622e-kserve-75f496cc64 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.HTTPRoute kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.Deployment kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.DestinationRule kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.Service kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-workload-svc
(x13)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-inline-config-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-inline-config-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-inline-config-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-inline-config-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-inline-config-test-kserve-workload-svc

kserve-ci-e2e-test

replicaset-controller

scheduler-inline-config-test-kserve-router-scheduler-95466f977

SuccessfulCreate

Created pod: scheduler-inline-config-test-kserve-router-scheduler-95466xddxz

kserve-ci-e2e-test

replicaset-controller

scheduler-inline-config-test-kserve-5748858bf9

SuccessfulCreate

Created pod: scheduler-inline-config-test-kserve-5748858bf9-zzm67

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-inline-config-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-inline-config-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-inline-config-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-inline-config-test-epp-service
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-inline-config-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-inline-config-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-inline-config-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-inline-config-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-5748858bf9-zzm67

Pulling

Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f"

kserve-ci-e2e-test

multus

scheduler-inline-config-test-kserve-5748858bf9-zzm67

AddedInterface

Add eth0 [10.134.0.18/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

scheduler-inline-config-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-inline-config-test-kserve-5748858bf9 from 0 to 1

kserve-ci-e2e-test

deployment-controller

scheduler-inline-config-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-inline-config-test-kserve-router-scheduler-95466f977 from 0 to 1

kserve-ci-e2e-test

multus

scheduler-inline-config-test-kserve-router-scheduler-95466xddxz

AddedInterface

Add eth0 [10.132.0.58/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-95466xddxz

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-95466xddxz

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-95466xddxz

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-75f496cc6428bk4

Killing

Stopping container storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-95466xddxz

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1"

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-5748858bf9-zzm67

Pulled

Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" in 3.375s (3.375s including waiting). Image size: 301469265 bytes.

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-5748858bf9-zzm67

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-95466xddxz

Pulling

Pulling image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1"

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-95466xddxz

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-95466xddxz

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-95466xddxz

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" in 1.429s (1.429s including waiting). Image size: 87241021 bytes.

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-5748858bf9-zzm67

Created

Created container: storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.DestinationRule kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-workload-svc

kserve-ci-e2e-test

deployment-controller

gw-section-name-router-with-gat-f1d92d0f-kserve

ScalingReplicaSet

Scaled up replica set gw-section-name-router-with-gat-f1d92d0f-kserve-746bf7d5f from 0 to 1

kserve-ci-e2e-test

multus

gw-section-name-router-with-gat-f1d92d0f-kserve-746bf7d5f-nzdcd

AddedInterface

Add eth0 [10.133.0.31/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

gw-section-name-router-with-gat-f1d92d0f-kserve-746bf7d5f

SuccessfulCreate

Created pod: gw-section-name-router-with-gat-f1d92d0f-kserve-746bf7d5f-nzdcd

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-746bf7d5f-nzdcd

Pulling

Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f"

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.HTTPRoute kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.Service kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.Deployment kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.Secret kserve-ci-e2e-test/gw-sec2774c263d49959f50d9eebc552e13bf9-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-746bf7d5f-nzdcd

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-746bf7d5f-nzdcd

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-746bf7d5f-nzdcd

Pulled

Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" in 3.5s (3.5s including waiting). Image size: 301469265 bytes.

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-746bf7d5f-nzdcd

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-5748858bf9-zzm67

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-746bf7d5f-nzdcd

Started

Started container main

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-746bf7d5f-nzdcd

Created

Created container: main

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-746bf7d5f-nzdcd

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.304s (1.304s including waiting). Image size: 98346788 bytes.

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-5748858bf9-zzm67

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-5748858bf9-zzm67

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-5748858bf9-zzm67

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.831s (1.831s including waiting). Image size: 98346788 bytes.

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-746bf7d5f-nzdcd

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-inline-config-test-kserve-self-signed-certs
(x13)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-6fa1027a

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-6fa1027a

Created

Created v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-route

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-6fa1027a-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-684c9d6b7 from 0 to 1

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-684c9d6b7-t4hcw

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-6fa1027a

Created

Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-6fa1027a

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-6fa1027a

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-6fa1027a

Created

Created v1.Secret kserve-ci-e2e-test/llmisvad71fa5348b85aebd404221bba611457-kserve-self-signed-certs

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-684c9d6b7-t4hcw

AddedInterface

Add eth0 [10.133.0.32/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-684c9d6b7

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-684c9d6b7-t4hcw

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-684c9d6b7-t4hcw

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-95466xddxz

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-95466xddxz

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-95466xddxz

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" in 31.11s (31.11s including waiting). Image size: 2907020949 bytes.

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-684c9d6b7-t4hcw

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-95466xddxz

Unhealthy

Startup probe failed: Get "http://10.132.0.58:8082/healthz": dial tcp 10.132.0.58:8082: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-6fa1027a

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisvad71fa5348b85aebd404221bba611457-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-95466xddxz

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-inline-config-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-5748858bf9-zzm67

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-95466xddxz

Killing

Stopping container tokenizer
(x13)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-configmap-ref-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-configmap-ref-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-configmap-ref-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-service

kserve-ci-e2e-test

deployment-controller

scheduler-configmap-ref-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-configmap-ref-test-kserve-59c9cf8475 from 0 to 1

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-59c9cf8475-gjmqb

Started

Started container storage-initializer
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-59c9cf8475-gjmqb

Created

Created container: storage-initializer

kserve-ci-e2e-test

replicaset-controller

scheduler-configmap-ref-test-kserve-59c9cf8475

SuccessfulCreate

Created pod: scheduler-configmap-ref-test-kserve-59c9cf8475-gjmqb

kserve-ci-e2e-test

replicaset-controller

scheduler-configmap-ref-test-kserve-router-scheduler-579b567ff9

SuccessfulCreate

Created pod: scheduler-configmap-ref-test-kserve-router-scheduler-579b56x6tg

kserve-ci-e2e-test

multus

scheduler-configmap-ref-test-kserve-router-scheduler-579b56x6tg

AddedInterface

Add eth0 [10.132.0.59/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-579b56x6tg

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-579b56x6tg

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-579b56x6tg

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-configmap-ref-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-self-signed-certs

kserve-ci-e2e-test

multus

scheduler-configmap-ref-test-kserve-59c9cf8475-gjmqb

AddedInterface

Add eth0 [10.134.0.19/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-59c9cf8475-gjmqb

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-rb

kserve-ci-e2e-test

deployment-controller

scheduler-configmap-ref-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-configmap-ref-test-kserve-router-scheduler-579b567ff9 from 0 to 1

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-579b56x6tg

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-579b56x6tg

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-579b56x6tg

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-579b56x6tg

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-579b56x6tg

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-579b56x6tg

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-59c9cf8475-gjmqb

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-59c9cf8475-gjmqb

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-59c9cf8475-gjmqb

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-configmap-ref-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-579b56x6tg

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-579b56x6tg

Killing

Stopping container tokenizer

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-59c9cf8475-gjmqb

Killing

Stopping container main
(x10)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-ha-replicas-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-ha-replicas-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-ha-replicas-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs

kserve-ci-e2e-test

replicaset-controller

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b85f

SuccessfulCreate

Created pod: scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8smmhl

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve

kserve-ci-e2e-test

replicaset-controller

scheduler-ha-replicas-test-kserve-865c7d9d55

SuccessfulCreate

Created pod: scheduler-ha-replicas-test-kserve-865c7d9d55-dgjrx

kserve-ci-e2e-test

replicaset-controller

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b85f

SuccessfulCreate

Created pod: scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8m4xzm

kserve-ci-e2e-test

deployment-controller

scheduler-ha-replicas-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-ha-replicas-test-kserve-865c7d9d55 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-ha-replicas-test-epp-auth-rb

kserve-ci-e2e-test

deployment-controller

scheduler-ha-replicas-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b85f from 0 to 2

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-rb

kserve-ci-e2e-test

multus

scheduler-ha-replicas-test-kserve-865c7d9d55-dgjrx

AddedInterface

Add eth0 [10.134.0.20/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8smmhl

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8m4xzm

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8m4xzm

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

multus

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8smmhl

AddedInterface

Add eth0 [10.134.0.21/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8smmhl

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8smmhl

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8m4xzm

Started

Started container storage-initializer
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-shadow-svc

kserve-ci-e2e-test

multus

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8m4xzm

AddedInterface

Add eth0 [10.132.0.60/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-865c7d9d55-dgjrx

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-865c7d9d55-dgjrx

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-865c7d9d55-dgjrx

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8smmhl

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1"

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8m4xzm

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8smmhl

Pulling

Pulling image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1"

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8smmhl

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8smmhl

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8smmhl

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" in 945ms (945ms including waiting). Image size: 87241021 bytes.

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8m4xzm

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8m4xzm

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8m4xzm

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8m4xzm

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8m4xzm

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-684c9d6b7-t4hcw

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-684c9d6b7-t4hcw

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-684c9d6b7-t4hcw

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8smmhl

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8smmhl

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8smmhl

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" in 28.54s (28.54s including waiting). Image size: 2907020949 bytes.

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-684c9d6b7-t4hcw

Killing

Stopping container main
(x2)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-684c9d6b7-t4hcw

FailedMount

MountVolume.SetUp failed for volume "tls-certs" : secret "llmisvad71fa5348b85aebd404221bba611457-kserve-self-signed-certs" not found
(x11)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-55f7ae4a

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Created

Created v1.Secret kserve-ci-e2e-test/llmisvdde380eaa9fe1facad32d45131f9e34d-kserve-self-signed-certs

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-5f4978c9bc

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-5f4978c9bcksn8t

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-5f4978c9bc from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-teste2c0c7deeb42f8b9846a7a45d2caecd2-llmisvc-mode

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheduler

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-5f4978c9bcksn8t

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schedx5gc

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

replicaset-controller

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheduler-677f64448

SuccessfulCreate

Created pod: llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schedx5gc

kserve-ci-e2e-test

kubelet

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schedx5gc

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schedx5gc

Started

Started container storage-initializer

kserve-ci-e2e-test

deployment-controller

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheduler-677f64448 from 0 to 1

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-5f4978c9bcksn8t

AddedInterface

Add eth0 [10.133.0.33/23] from ovn-kubernetes

kserve-ci-e2e-test

multus

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schedx5gc

AddedInterface

Add eth0 [10.132.0.61/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-epp-rb

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-5f4978c9bcksn8t

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-5f4978c9bcksn8t

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schedx5gc

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schedx5gc

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schedx5gc

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schedx5gc

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schedx5gc

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schedx5gc

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-5f4978c9bcksn8t

Pulling

Pulling image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0"

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisvdde380eaa9fe1facad32d45131f9e34d-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-865c7d9d55-dgjrx

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-865c7d9d55-dgjrx

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-865c7d9d55-dgjrx

Created

Created container: main
(x7)

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8smmhl

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-ha-replicas-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8m4xzm

Killing

Stopping container tokenizer

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8smmhl

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8smmhl

Killing

Stopping container tokenizer

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8m4xzm

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-865c7d9d55-dgjrx

Killing

Stopping container main
(x2)

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-865c7d9d55-dgjrx

FailedMount

MountVolume.SetUp failed for volume "tls-certs" : secret "scheduler-ha-replicas-test-kserve-self-signed-certs" not found

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6cb95b8m4xzm

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.132.0.60:9003" within 1s: context deadline exceeded
(x13)

kserve-ci-e2e-test

OpenDataHubModelController

precise-prefix-cache-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy precise-prefix-cache-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/precise-prefix-cache-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "precise-prefix-cache-test-kserve-route" not found

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-5f4978c9bcksn8t

Pulled

Successfully pulled image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" in 46.18s (46.18s including waiting). Image size: 3531177328 bytes.

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-5f4978c9bcksn8t

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-5f4978c9bcksn8t

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Secret kserve-ci-e2e-test/precise-prefix-cache-test-kserve-self-signed-certs

kserve-ci-e2e-test

deployment-controller

precise-prefix-cache-test-kserve

ScalingReplicaSet

Scaled up replica set precise-prefix-cache-test-kserve-7cbfd5474f from 0 to 2

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Service kserve-ci-e2e-test/precise-prefix-cache-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Deployment kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7cbfd5474f-smcw5

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7cbfd5474f-smcw5

Created

Created container: storage-initializer

kserve-ci-e2e-test

replicaset-controller

precise-prefix-cache-test-kserve-7cbfd5474f

SuccessfulCreate

Created pod: precise-prefix-cache-test-kserve-7cbfd5474f-6xw9c

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-precise-prefix-cache-test-epp-auth-rb

kserve-ci-e2e-test

multus

precise-prefix-cache-test-kserve-7cbfd5474f-smcw5

AddedInterface

Add eth0 [10.134.0.22/23] from ovn-kubernetes

kserve-ci-e2e-test

multus

precise-prefix-cache-test-kserve-router-scheduler-7dbf97ff6l2d7

AddedInterface

Add eth0 [10.134.0.23/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-7dbf97ff6l2d7

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-7dbf97ff6l2d7

Created

Created container: storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/precise-prefix-cache-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Service kserve-ci-e2e-test/precise-prefix-cache-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Deployment kserve-ci-e2e-test/precise-prefix-cache-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/precise-prefix-cache-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Role kserve-ci-e2e-test/precise-prefix-cache-test-epp-role

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7cbfd5474f-smcw5

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

replicaset-controller

precise-prefix-cache-test-kserve-7cbfd5474f

SuccessfulCreate

Created pod: precise-prefix-cache-test-kserve-7cbfd5474f-smcw5

kserve-ci-e2e-test

replicaset-controller

precise-prefix-cache-test-kserve-router-scheduler-7dbf97fff7

SuccessfulCreate

Created pod: precise-prefix-cache-test-kserve-router-scheduler-7dbf97ff6l2d7

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7cbfd5474f-6xw9c

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7cbfd5474f-6xw9c

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7cbfd5474f-6xw9c

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

multus

precise-prefix-cache-test-kserve-7cbfd5474f-6xw9c

AddedInterface

Add eth0 [10.133.0.34/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

precise-prefix-cache-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set precise-prefix-cache-test-kserve-router-scheduler-7dbf97fff7 from 0 to 1

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-7dbf97ff6l2d7

Created

Created container: main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-7dbf97ff6l2d7

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-7dbf97ff6l2d7

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-7dbf97ff6l2d7

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-7dbf97ff6l2d7

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-7dbf97ff6l2d7

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-7dbf97ff6l2d7

Started

Started container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7cbfd5474f-smcw5

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7cbfd5474f-smcw5

Created

Created container: main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7cbfd5474f-6xw9c

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7cbfd5474f-6xw9c

Created

Created container: main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7cbfd5474f-6xw9c

Started

Started container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7cbfd5474f-smcw5

Started

Started container main
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/precise-prefix-cache-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Updated

Updated v1.Secret kserve-ci-e2e-test/precise-prefix-cache-test-kserve-self-signed-certs
(x2)

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-7dbf97ff6l2d7

Unhealthy

Liveness probe failed: timeout: failed to connect service "10.134.0.23:9003" within 1s: context deadline exceeded

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-precise-prefix-cache-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7cbfd5474f-6xw9c

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-7dbf97ff6l2d7

Killing

Stopping container tokenizer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7cbfd5474f-smcw5

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-7dbf97ff6l2d7

Killing

Stopping container main

kserve-ci-e2e-test

replicaset-controller

conv-test-round-trip-kserve-56766c79cf

SuccessfulCreate

Created pod: conv-test-round-trip-kserve-56766c79cf-wgb7x

kserve-ci-e2e-test

LLMInferenceServiceController

conv-test-round-trip

Created

Created v1.Deployment kserve-ci-e2e-test/conv-test-round-trip-kserve

kserve-ci-e2e-test

kubelet

conv-test-round-trip-kserve-56766c79cf-wgb7x

Started

Started container storage-initializer

kserve-ci-e2e-test

multus

conv-test-round-trip-kserve-56766c79cf-wgb7x

AddedInterface

Add eth0 [10.133.0.35/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

conv-test-round-trip-kserve

ScalingReplicaSet

Scaled up replica set conv-test-round-trip-kserve-56766c79cf from 0 to 1

kserve-ci-e2e-test

kubelet

conv-test-round-trip-kserve-56766c79cf-wgb7x

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

conv-test-round-trip

Created

Created v1.Secret kserve-ci-e2e-test/conv-test-round-trip-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

conv-test-round-trip-kserve-56766c79cf-wgb7x

Created

Created container: storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

conv-test-round-trip

Created

Created v1.Service kserve-ci-e2e-test/conv-test-round-trip-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

conv-test-round-trip

Created

Created v1.HTTPRoute kserve-ci-e2e-test/conv-test-round-trip-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

conv-test-round-trip

Created

Created v1.DestinationRule kserve-ci-e2e-test/conv-test-round-trip-kserve-workload-svc

kserve-ci-e2e-test

kubelet

conv-test-round-trip-kserve-56766c79cf-wgb7x

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

conv-test-round-trip-kserve-56766c79cf-wgb7x

Created

Created container: main

kserve-ci-e2e-test

kubelet

conv-test-round-trip-kserve-56766c79cf-wgb7x

Started

Started container main

kserve-ci-e2e-test

multus

stop-feature-test-kserve-router-scheduler-6944855df4-f4h99

AddedInterface

Add eth0 [10.134.0.24/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/stop-feature-test-epp-rb

kserve-ci-e2e-test

replicaset-controller

stop-feature-test-kserve-router-scheduler-6944855df4

SuccessfulCreate

Created pod: stop-feature-test-kserve-router-scheduler-6944855df4-f4h99

kserve-ci-e2e-test

deployment-controller

stop-feature-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set stop-feature-test-kserve-router-scheduler-6944855df4 from 0 to 1

kserve-ci-e2e-test

deployment-controller

stop-feature-test-kserve

ScalingReplicaSet

Scaled up replica set stop-feature-test-kserve-96dfd8dc5 from 0 to 1

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-6944855df4-f4h99

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

replicaset-controller

stop-feature-test-kserve-96dfd8dc5

SuccessfulCreate

Created pod: stop-feature-test-kserve-96dfd8dc5-2gs2q

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs

kserve-ci-e2e-test

multus

stop-feature-test-kserve-96dfd8dc5-2gs2q

AddedInterface

Add eth0 [10.133.0.36/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-96dfd8dc5-2gs2q

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-96dfd8dc5-2gs2q

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-96dfd8dc5-2gs2q

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Service kserve-ci-e2e-test/stop-feature-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/stop-feature-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-stop-feature-test-epp-auth-rb
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/stop-feature-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Service kserve-ci-e2e-test/stop-feature-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Role kserve-ci-e2e-test/stop-feature-test-epp-role

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-6944855df4-f4h99

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-6944855df4-f4h99

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-6944855df4-f4h99

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-6944855df4-f4h99

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-6944855df4-f4h99

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-6944855df4-f4h99

Started

Started container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-6944855df4-f4h99

Created

Created container: main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-6944855df4-f4h99

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

conv-test-round-trip-kserve-56766c79cf-wgb7x

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Updated

Updated v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-96dfd8dc5-2gs2q

Created

Created container: main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-96dfd8dc5-2gs2q

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-96dfd8dc5-2gs2q

Started

Started container main
(x9)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-5f4978c9bcksn8t

Unhealthy

Startup probe failed: Get "https://10.133.0.33:8000/health": dial tcp 10.133.0.33:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-5f4978c9bcksn8t

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-55f7ae4a

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-teste2c0c7deeb42f8b9846a7a45d2caecd2-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schedx5gc

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schedx5gc

Killing

Stopping container tokenizer
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

custom-route-timeout-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy custom-route-timeout-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/custom-route-timeout-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "custom-route-timeout-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Service kserve-ci-e2e-test/custom-route-timeout-test-kserve-workload-svc

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-test-kserve-6bb559dc87

SuccessfulCreate

Created pod: custom-route-timeout-test-kserve-6bb559dc87-6tns4

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-test-kserve

ScalingReplicaSet

Scaled up replica set custom-route-timeout-test-kserve-6bb559dc87 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Secret kserve-ci-e2e-test/custom-route-timeout-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-6bb559dc87-6tns4

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/custom-route-timeout-test-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-5c699bbd8m7bx

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-5c699bbd8m7bx

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Role kserve-ci-e2e-test/custom-route-timeout-test-epp-role

kserve-ci-e2e-test

multus

custom-route-timeout-test-kserve-router-scheduler-5c699bbd8m7bx

AddedInterface

Add eth0 [10.132.0.62/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-6bb559dc87-6tns4

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/custom-route-timeout-test-epp-rb

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-6bb559dc87-6tns4

Created

Created container: storage-initializer

kserve-ci-e2e-test

multus

custom-route-timeout-test-kserve-6bb559dc87-6tns4

AddedInterface

Add eth0 [10.133.0.37/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-test-kserve-router-scheduler-5c699bbdd8

SuccessfulCreate

Created pod: custom-route-timeout-test-kserve-router-scheduler-5c699bbd8m7bx

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set custom-route-timeout-test-kserve-router-scheduler-5c699bbdd8 from 0 to 1

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-5c699bbd8m7bx

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Service kserve-ci-e2e-test/custom-route-timeout-test-epp-service

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-5c699bbd8m7bx

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-5c699bbd8m7bx

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-5c699bbd8m7bx

Started

Started container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-5c699bbd8m7bx

Created

Created container: main

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-5c699bbd8m7bx

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-5c699bbd8m7bx

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-6bb559dc87-6tns4

Started

Started container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-6bb559dc87-6tns4

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-6bb559dc87-6tns4

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Updated

Updated v1.Secret kserve-ci-e2e-test/custom-route-timeout-test-kserve-self-signed-certs
(x10)

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-96dfd8dc5-2gs2q

Unhealthy

Startup probe failed: Get "https://10.133.0.36:8000/health": dial tcp 10.133.0.36:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-96dfd8dc5-2gs2q

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.ServiceAccount kserve-ci-e2e-test/stop-feature-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Service kserve-ci-e2e-test/stop-feature-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Error

Reconciliation failed: failed to reconcile networking: failed to reconcile scheduler: failed to build expected scheduler deployment: failed to attach model artifacts to scheduler deployment: secrets "stop-feature-test-epp-sa-dockercfg-qf8fj" not found

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-6944855df4-f4h99

Killing

Stopping container tokenizer

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.InferencePool kserve-ci-e2e-test/stop-feature-test-inference-pool

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-6944855df4-f4h99

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1alpha2.InferencePool kserve-ci-e2e-test/stop-feature-test-inference-pool

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.HTTPRoute kserve-ci-e2e-test/stop-feature-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Service kserve-ci-e2e-test/stop-feature-test-epp-service
(x24)

kserve-ci-e2e-test

OpenDataHubModelController

stop-feature-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy stop-feature-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/stop-feature-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "stop-feature-test-kserve-route" not found

kserve-ci-e2e-test

replicaset-controller

stop-feature-test-kserve-96dfd8dc5

SuccessfulCreate

Created pod: stop-feature-test-kserve-96dfd8dc5-2rrzb

kserve-ci-e2e-test

deployment-controller

stop-feature-test-kserve

ScalingReplicaSet

Scaled up replica set stop-feature-test-kserve-96dfd8dc5 from 0 to 1

kserve-ci-e2e-test

multus

stop-feature-test-kserve-96dfd8dc5-2rrzb

AddedInterface

Add eth0 [10.133.0.38/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-96dfd8dc5-2rrzb

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-96dfd8dc5-2rrzb

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-6944855df4-q7mvb

Started

Started container storage-initializer

kserve-ci-e2e-test

deployment-controller

stop-feature-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set stop-feature-test-kserve-router-scheduler-6944855df4 from 0 to 1

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-6944855df4-q7mvb

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-96dfd8dc5-2rrzb

Created

Created container: storage-initializer

kserve-ci-e2e-test

multus

stop-feature-test-kserve-router-scheduler-6944855df4-q7mvb

AddedInterface

Add eth0 [10.134.0.25/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-6944855df4-q7mvb

Created

Created container: storage-initializer

kserve-ci-e2e-test

replicaset-controller

stop-feature-test-kserve-router-scheduler-6944855df4

SuccessfulCreate

Created pod: stop-feature-test-kserve-router-scheduler-6944855df4-q7mvb

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-6944855df4-q7mvb

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-6944855df4-q7mvb

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-6944855df4-q7mvb

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-6944855df4-q7mvb

Started

Started container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-6944855df4-q7mvb

Created

Created container: main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-6944855df4-q7mvb

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine
(x10)

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-6bb559dc87-6tns4

Unhealthy

Startup probe failed: Get "https://10.133.0.37:8000/health": dial tcp 10.133.0.37:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-5c699bbd8m7bx

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-5c699bbd8m7bx

Killing

Stopping container tokenizer

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-6bb559dc87-6tns4

Killing

Stopping container main
(x4)

kserve-ci-e2e-test

gateway_labeler_controller

router-gateway-1

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway router-gateway-1

kserve-ci-e2e-test

deployment-controller

router-with-refs-test-kserve

ScalingReplicaSet

Scaled up replica set router-with-refs-test-kserve-fcd884cb7 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Deployment kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Deployment kserve-ci-e2e-test/router-with-refs-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Secret kserve-ci-e2e-test/router-with-refs-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-test-epp-sa

kserve-ci-e2e-test

deployment-controller

router-with-refs-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set router-with-refs-test-kserve-router-scheduler-6dbd7f8f9b from 0 to 1

kserve-ci-e2e-test

replicaset-controller

router-with-refs-test-kserve-router-scheduler-6dbd7f8f9b

SuccessfulCreate

Created pod: router-with-refs-test-kserve-router-scheduler-6dbd7f8f9b-tdmjc

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Service kserve-ci-e2e-test/router-with-refs-test-kserve-workload-svc

kserve-ci-e2e-test

replicaset-controller

router-with-refs-test-kserve-fcd884cb7

SuccessfulCreate

Created pod: router-with-refs-test-kserve-fcd884cb7-x6msw

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/router-with-refs-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-fcd884cb7-x6msw

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Role kserve-ci-e2e-test/router-with-refs-test-epp-role

kserve-ci-e2e-test

multus

router-with-refs-test-kserve-fcd884cb7-x6msw

AddedInterface

Add eth0 [10.133.0.39/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-fcd884cb7-x6msw

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-fcd884cb7-x6msw

Created

Created container: storage-initializer
(x5)

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/router-with-refs-test-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-6dbd7f8f9b-tdmjc

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-6dbd7f8f9b-tdmjc

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-6dbd7f8f9b-tdmjc

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

multus

router-with-refs-test-kserve-router-scheduler-6dbd7f8f9b-tdmjc

AddedInterface

Add eth0 [10.132.0.63/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Service kserve-ci-e2e-test/router-with-refs-test-epp-service

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-6dbd7f8f9b-tdmjc

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-6dbd7f8f9b-tdmjc

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-6dbd7f8f9b-tdmjc

Created

Created container: main

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-6dbd7f8f9b-tdmjc

Started

Started container main

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-6dbd7f8f9b-tdmjc

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-6dbd7f8f9b-tdmjc

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-fcd884cb7-x6msw

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Updated

Updated v1.Secret kserve-ci-e2e-test/router-with-refs-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-fcd884cb7-x6msw

Created

Created container: main

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-fcd884cb7-x6msw

Started

Started container main
(x9)

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-fcd884cb7-x6msw

Unhealthy

Startup probe failed: Get "https://10.133.0.39:8000/health": dial tcp 10.133.0.39:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-fcd884cb7-x6msw

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-6dbd7f8f9b-tdmjc

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-6dbd7f8f9b-tdmjc

Killing

Stopping container tokenizer
(x11)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-05aa9bba

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-route" not found

kserve-ci-e2e-test

deployment-controller

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheduler-7c6fc67c6 from 0 to 1

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-f8dd46d4b from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test4ce8108fb3d8aff064c322bee6e173f5-llmisvc-mode

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-role

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-f8dd46d4b-f8l57

AddedInterface

Add eth0 [10.133.0.40/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Created

Created v1.Secret kserve-ci-e2e-test/llmisv8f1a6f044e8c7a4d31a250e0c4861caf-kserve-self-signed-certs

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6fcd8d5c7 from 0 to 1

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-f8dd46d4b-f8l57

Pulling

Pulling image "ghcr.io/llm-d/llm-d-routing-sidecar:v0.7.1"

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6fcl8vk

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-rb

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-f8dd46d4b

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-f8dd46d4b-f8l57

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6fcd8d5c7

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6fcl8vk

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6fcl8vk

AddedInterface

Add eth0 [10.133.0.41/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheduler-7c6fc67c6

SuccessfulCreate

Created pod: llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schedv898

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6fcl8vk

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6fcl8vk

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-f8dd46d4b-f8l57

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-routing-sidecar:v0.7.1" in 1.056s (1.056s including waiting). Image size: 67767940 bytes.

kserve-ci-e2e-test

kubelet

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schedv898

Started

Started container storage-initializer
(x10)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schedv898

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schedv898

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-f8dd46d4b-f8l57

Created

Created container: llm-d-routing-sidecar

kserve-ci-e2e-test

multus

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schedv898

AddedInterface

Add eth0 [10.132.0.64/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-f8dd46d4b-f8l57

Started

Started container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-f8dd46d4b-f8l57

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-f8dd46d4b-f8l57

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schedv898

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schedv898

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-f8dd46d4b-f8l57

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schedv898

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schedv898

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schedv898

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schedv898

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6fcl8vk

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6fcl8vk

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6fcl8vk

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-f8dd46d4b-f8l57

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-f8dd46d4b-f8l57

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-f8dd46d4b-f8l57

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv8f1a6f044e8c7a4d31a250e0c4861caf-kserve-self-signed-certs
(x14)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-f8dd46d4b-f8l57

Unhealthy

Startup probe failed: Get "https://10.133.0.40:8001/health": dial tcp 10.133.0.40:8001: connect: connection refused
(x14)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6fcl8vk

Unhealthy

Startup probe failed: Get "https://10.133.0.41:8000/health": dial tcp 10.133.0.41:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-05aa9bba

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test4ce8108fb3d8aff064c322bee6e173f5-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-6fcl8vk

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-f8dd46d4b-f8l57

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-f8dd46d4b-f8l57

Killing

Stopping container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schedv898

Killing

Stopping container tokenizer

kserve-ci-e2e-test

kubelet

llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schedv898

Killing

Stopping container main
(x11)

kserve-ci-e2e-test

OpenDataHubModelController

custom-route-timeout-pd-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy custom-route-timeout-pd-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "custom-route-timeout-pd-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Role kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-role

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-pd-test-kserve-c86f76cb9

SuccessfulCreate

Created pod: custom-route-timeout-pd-test-kserve-c86f76cb9-dcj46

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-pd-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-pd-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Secret kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Service kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-workload-svc

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-pd-test-kserve-prefill-75d85d5cb8

SuccessfulCreate

Created pod: custom-route-timeout-pd-test-kserve-prefill-75d85d5cb8-crdgj

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-pd-test-kserve-prefill

ScalingReplicaSet

Scaled up replica set custom-route-timeout-pd-test-kserve-prefill-75d85d5cb8 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-rb

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-pd-test-kserve

ScalingReplicaSet

Scaled up replica set custom-route-timeout-pd-test-kserve-c86f76cb9 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-pd-test-kserve-router-scheduler-5cb7fdf67

SuccessfulCreate

Created pod: custom-route-timeout-pd-test-kserve-router-scheduler-5cb7f48br2

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-5cb7f48br2

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-c86f76cb9-dcj46

Pulled

Container image "ghcr.io/llm-d/llm-d-routing-sidecar:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-c86f76cb9-dcj46

Created

Created container: llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-c86f76cb9-dcj46

Started

Started container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-5cb7f48br2

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-5cb7f48br2

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-75d85d5cb8-crdgj

Started

Started container storage-initializer

kserve-ci-e2e-test

multus

custom-route-timeout-pd-test-kserve-router-scheduler-5cb7f48br2

AddedInterface

Add eth0 [10.132.0.65/23] from ovn-kubernetes
(x10)

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-shadow-svc

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-pd-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set custom-route-timeout-pd-test-kserve-router-scheduler-5cb7fdf67 from 0 to 1

kserve-ci-e2e-test

multus

custom-route-timeout-pd-test-kserve-c86f76cb9-dcj46

AddedInterface

Add eth0 [10.133.0.42/23] from ovn-kubernetes

kserve-ci-e2e-test

multus

custom-route-timeout-pd-test-kserve-prefill-75d85d5cb8-crdgj

AddedInterface

Add eth0 [10.133.0.43/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-75d85d5cb8-crdgj

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-75d85d5cb8-crdgj

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-c86f76cb9-dcj46

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-c86f76cb9-dcj46

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-c86f76cb9-dcj46

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-5cb7f48br2

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-5cb7f48br2

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-5cb7f48br2

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-5cb7f48br2

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-5cb7f48br2

Started

Started container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-5cb7f48br2

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Updated

Updated v1.Secret kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-75d85d5cb8-crdgj

Created

Created container: main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-75d85d5cb8-crdgj

Started

Started container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-75d85d5cb8-crdgj

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-c86f76cb9-dcj46

Started

Started container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-c86f76cb9-dcj46

Created

Created container: main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-c86f76cb9-dcj46

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine
(x10)

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-75d85d5cb8-crdgj

Unhealthy

Startup probe failed: Get "https://10.133.0.43:8000/health": dial tcp 10.133.0.43:8000: connect: connection refused
(x9)

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-c86f76cb9-dcj46

Unhealthy

Startup probe failed: Get "https://10.133.0.42:8001/health": dial tcp 10.133.0.42:8001: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-pd-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-7bb57db77b-x7rzn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:d518f3d1539f45e1253c5c9fa22062802804601d4998cd50344e476a3cc388fe" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-75d85d5cb8-crdgj

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-5cb7f48br2

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-5cb7f48br2

Killing

Stopping container tokenizer

kserve-ci-e2e-test

multus

router-gateway-2-openshift-default-7bb57db77b-x7rzn

AddedInterface

Add eth0 [10.132.0.66/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-7bb57db77b-x7rzn

Created

Created container: istio-proxy

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-7bb57db77b-x7rzn

Started

Started container istio-proxy

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-c86f76cb9-dcj46

Killing

Stopping container llm-d-routing-sidecar

kserve-ci-e2e-test

replicaset-controller

router-gateway-2-openshift-default-7bb57db77b

SuccessfulCreate

Created pod: router-gateway-2-openshift-default-7bb57db77b-x7rzn

kserve-ci-e2e-test

deployment-controller

router-gateway-2-openshift-default

ScalingReplicaSet

Scaled up replica set router-gateway-2-openshift-default-7bb57db77b from 0 to 1

kserve-ci-e2e-test

service-controller

router-gateway-2-openshift-default

EnsuringLoadBalancer

Ensuring load balancer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-c86f76cb9-dcj46

Killing

Stopping container main

kserve-ci-e2e-test

gateway_labeler_controller

router-gateway-2

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway router-gateway-2

kserve-ci-e2e-test

service-controller

router-gateway-2-openshift-default

EnsuredLoadBalancer

Ensured load balancer

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-pd-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-pd-test-epp-auth-rb

kserve-ci-e2e-test

deployment-controller

router-with-refs-pd-test-kserve

ScalingReplicaSet

Scaled up replica set router-with-refs-pd-test-kserve-8c9cd8f from 0 to 1

kserve-ci-e2e-test

deployment-controller

router-with-refs-pd-test-kserve-prefill

ScalingReplicaSet

Scaled up replica set router-with-refs-pd-test-kserve-prefill-8597c688bd from 0 to 1

kserve-ci-e2e-test

replicaset-controller

router-with-refs-pd-test-kserve-prefill-8597c688bd

SuccessfulCreate

Created pod: router-with-refs-pd-test-kserve-prefill-8597c688bd-wbhgp

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Secret kserve-ci-e2e-test/router-with-refs-pd-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Role kserve-ci-e2e-test/router-with-refs-pd-test-kserve-role

kserve-ci-e2e-test

replicaset-controller

router-with-refs-pd-test-kserve-8c9cd8f

SuccessfulCreate

Created pod: router-with-refs-pd-test-kserve-8c9cd8f-v6xzj

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/router-with-refs-pd-test-kserve-rb

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Deployment kserve-ci-e2e-test/router-with-refs-pd-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Deployment kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Service kserve-ci-e2e-test/router-with-refs-pd-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-pd-test-epp-sa

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-8597c688bd-wbhgp

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-55f6fbbcfxwgqm

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-55f6fbbcfxwgqm

Created

Created container: storage-initializer

kserve-ci-e2e-test

multus

router-with-refs-pd-test-kserve-router-scheduler-55f6fbbcfxwgqm

AddedInterface

Add eth0 [10.132.0.67/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

router-with-refs-pd-test-kserve-router-scheduler-55f6fbbcf9

SuccessfulCreate

Created pod: router-with-refs-pd-test-kserve-router-scheduler-55f6fbbcfxwgqm

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-8c9cd8f-v6xzj

Started

Started container llm-d-routing-sidecar

kserve-ci-e2e-test

multus

router-with-refs-pd-test-kserve-prefill-8597c688bd-wbhgp

AddedInterface

Add eth0 [10.133.0.45/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-8597c688bd-wbhgp

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-55f6fbbcfxwgqm

Started

Started container storage-initializer
(x9)

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/router-with-refs-pd-test-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-8597c688bd-wbhgp

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-8c9cd8f-v6xzj

Pulled

Container image "ghcr.io/llm-d/llm-d-routing-sidecar:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-8c9cd8f-v6xzj

Created

Created container: llm-d-routing-sidecar

kserve-ci-e2e-test

multus

router-with-refs-pd-test-kserve-8c9cd8f-v6xzj

AddedInterface

Add eth0 [10.133.0.44/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

router-with-refs-pd-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set router-with-refs-pd-test-kserve-router-scheduler-55f6fbbcf9 from 0 to 1

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-8c9cd8f-v6xzj

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-8c9cd8f-v6xzj

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-8c9cd8f-v6xzj

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-55f6fbbcfxwgqm

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-55f6fbbcfxwgqm

Created

Created container: main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-55f6fbbcfxwgqm

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-55f6fbbcfxwgqm

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-55f6fbbcfxwgqm

Started

Started container main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-55f6fbbcfxwgqm

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-8597c688bd-wbhgp

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-8597c688bd-wbhgp

Started

Started container main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-8597c688bd-wbhgp

Created

Created container: main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-8c9cd8f-v6xzj

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-8c9cd8f-v6xzj

Created

Created container: main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-8c9cd8f-v6xzj

Started

Started container main
(x2)

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Updated

Updated v1.Secret kserve-ci-e2e-test/router-with-refs-pd-test-kserve-self-signed-certs
(x14)

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-8597c688bd-wbhgp

Unhealthy

Startup probe failed: Get "https://10.133.0.45:8000/health": dial tcp 10.133.0.45:8000: connect: connection refused
(x14)

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-8c9cd8f-v6xzj

Unhealthy

Startup probe failed: Get "https://10.133.0.44:8001/health": dial tcp 10.133.0.44:8001: connect: connection refused

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-55f6fbbcfxwgqm

Killing

Stopping container tokenizer

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-pd-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-8597c688bd-wbhgp

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-55f6fbbcfxwgqm

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-8c9cd8f-v6xzj

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-8c9cd8f-v6xzj

Killing

Stopping container llm-d-routing-sidecar

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-02ac88e7

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-workload-svc
(x10)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-02ac88e7

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-02ac88e7

Created

Created v1.Secret kserve-ci-e2e-test/llmisv8de1d74aab16d9cabd8b5aafeb5248e8-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-02ac88e7

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-6fdc6f5f95 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-02ac88e7

Created

Created v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-02ac88e7

Created

Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-workload-svc

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-6fdc6f5f95

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-6fdc6f5f95t7rxc

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-6fdc6f5f95t7rxc

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-6fdc6f5f95t7rxc

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-6fdc6f5f95t7rxc

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-6fdc6f5f95t7rxc

AddedInterface

Add eth0 [10.133.0.46/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-6fdc6f5f95t7rxc

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-6fdc6f5f95t7rxc

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-6fdc6f5f95t7rxc

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-02ac88e7

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv8de1d74aab16d9cabd8b5aafeb5248e8-kserve-self-signed-certs
(x10)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-6fdc6f5f95t7rxc

Unhealthy

Startup probe failed: Get "https://10.133.0.46:8000/health": dial tcp 10.133.0.46:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-6fdc6f5f95t7rxc

Killing

Stopping container main
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-f312f5ec

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Created

Created v1.Secret kserve-ci-e2e-test/llmisv4e643bc258191ffc517a31cd1d0ddd27-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-scc

kserve-ci-e2e-test

replicaset-controller

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-scheduler-6c47cf69c6

SuccessfulCreate

Created pod: llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegkglh

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Created

Created v1.LeaderWorkerSet kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegkglh

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegkglh

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegkglh

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

multus

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegkglh

AddedInterface

Add eth0 [10.132.0.69/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-scheduler-6c47cf69c6 from 0 to 1

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1

Started

Started container storage-initializer
(x8)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-epp-role

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1

AddedInterface

Add eth0 [10.132.0.68/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-workload-svc

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:0459e31fff147b1f9487117ac00a86221572ce2251d3068d36cf76ed2126ea4f" already present on machine

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn

CreatingRevision

Creating revision with key 594568cbcb for a newly created LeaderWorkerSet

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn

GroupsProgressing

Created leader statefulset llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn
(x2)

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn

GroupsProgressing

Replicas are progressing, with 0 groups ready of total 1 groups

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-testc9aefdf01a18c3acb55ce247710ebb71-llmisvc-mode

kserve-ci-e2e-test

statefulset-controller

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

SuccessfulCreate

create Pod llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1 in StatefulSet llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0 successful

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

AddedInterface

Add eth0 [10.133.0.47/23] from ovn-kubernetes

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn

GroupsProgressing

Created worker statefulset for leader pod llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

kserve-ci-e2e-test

statefulset-controller

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn

SuccessfulCreate

create Pod llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0 in StatefulSet llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn successful

kserve-ci-e2e-test

kubelet

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegkglh

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegkglh

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegkglh

Pulled

Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegkglh

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegkglh

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegkglh

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv4e643bc258191ffc517a31cd1d0ddd27-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1

Pulling

Pulling image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0"

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1

Pulled

Successfully pulled image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" in 27.68s (27.68s including waiting). Image size: 3531177328 bytes.

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1

Started

Started container main
(x9)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

Unhealthy

Startup probe failed: Get "https://10.133.0.47:8000/health": dial tcp 10.133.0.47:8000: connect: connection refused
(x2)

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn

AllGroupsReady

All replicas are ready, with 1 groups ready of total 1 groups

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegkglh

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-f312f5ec

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-testc9aefdf01a18c3acb55ce247710ebb71-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegkglh

Killing

Stopping container tokenizer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1

Killing

Stopping container main