Time Namespace Component RelatedObject Reason Message

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-7dfc79f5dc-wvz4n

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-7dfc79f5dc-wvz4n to ip-10-0-128-142.ec2.internal

kuadrant-system

authorino-b6c89bb85-6smtk

Scheduled

Successfully assigned kuadrant-system/authorino-b6c89bb85-6smtk to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4w2k6k

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4w2k6k to ip-10-0-142-172.ec2.internal

kserve-ci-e2e-test

prestop-hook-test-kserve-5b7d7d79ff-k79hx

Scheduled

Successfully assigned kserve-ci-e2e-test/prestop-hook-test-kserve-5b7d7d79ff-k79hx to ip-10-0-130-182.ec2.internal

kuadrant-system

kuadrant-console-plugin-85fd4b8b9d-28qld

Scheduled

Successfully assigned kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-28qld to ip-10-0-130-182.ec2.internal

kuadrant-system

authorino-operator-6d85f6564-hp27q

Scheduled

Successfully assigned kuadrant-system/authorino-operator-6d85f6564-hp27q to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

prestop-hook-test-kserve-router-scheduler-767d7586b8-gptkj

Scheduled

Successfully assigned kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-767d7586b8-gptkj to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

scheduler-v06-threshold-migration-test-kserve-prefill-5fc5n8sgm

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5n8sgm to ip-10-0-142-172.ec2.internal

kserve-ci-e2e-test

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc526tch

Scheduled

Successfully assigned kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc526tch to ip-10-0-130-182.ec2.internal

kuadrant-system

authorino-8df697c9f-zjlcz

Scheduled

Successfully assigned kuadrant-system/authorino-8df697c9f-zjlcz to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1 to ip-10-0-130-182.ec2.internal

openshift-ingress

router-default-5658d689b9-4zvzx

Scheduled

Successfully assigned openshift-ingress/router-default-5658d689b9-4zvzx to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-pd-test-kserve-67588db874-f6b8t

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-67588db874-f6b8t to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-test-kserve-router-scheduler-664966fctvzts

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-664966fctvzts to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwm7dq

Scheduled

Successfully assigned kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwm7dq to ip-10-0-142-172.ec2.internal

kuadrant-system

limitador-limitador-ff68ff5cb-lx9ww

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-ff68ff5cb-lx9ww to ip-10-0-128-142.ec2.internal

kuadrant-system

dns-operator-controller-manager-65b49595d7-dx5b4

Scheduled

Successfully assigned kuadrant-system/dns-operator-controller-manager-65b49595d7-dx5b4 to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

scheduler-ha-replicas-test-kserve-5c5cfcf776-jlqwh

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-jlqwh to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d9dg95

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d9dg95 to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dkvvq9

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dkvvq9 to ip-10-0-142-172.ec2.internal

kserve-ci-e2e-test

scheduler-inline-config-test-kserve-7c6c58cc96-rclxb

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-rclxb to ip-10-0-142-172.ec2.internal

kserve-ci-e2e-test

lora-single-adapter-test-kserve-75fdddbc5d-r8z7x

Scheduled

Successfully assigned kserve-ci-e2e-test/lora-single-adapter-test-kserve-75fdddbc5d-r8z7x to ip-10-0-128-142.ec2.internal

kuadrant-system

authorino-7c77957575-lkzck

Scheduled

Successfully assigned kuadrant-system/authorino-7c77957575-lkzck to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

rolling-upgrade-test-kserve-54bc76f55b-84gdc

Scheduled

Successfully assigned kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-84gdc to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

tls-verification-test-kserve-router-scheduler-5d7755bf56-kb4bv

Scheduled

Successfully assigned kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5d7755bf56-kb4bv to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

tls-verification-test-kserve-5847b7959f-z68fl

Scheduled

Successfully assigned kserve-ci-e2e-test/tls-verification-test-kserve-5847b7959f-z68fl to ip-10-0-128-142.ec2.internal

kuadrant-system

limitador-limitador-69574b596d-gxr6q

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-69574b596d-gxr6q to ip-10-0-142-172.ec2.internal

kserve-ci-e2e-test

rolling-upgrade-test-kserve-859df57bcb-7zscs

Scheduled

Successfully assigned kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-7zscs to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fmxwss

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fmxwss to ip-10-0-142-172.ec2.internal

kserve-ci-e2e-test

llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-795tshv

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-795tshv to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

scheduler-custom-template-test-kserve-router-scheduler-5677kqnn

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-5677kqnn to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddk5j4w

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddk5j4w to ip-10-0-142-172.ec2.internal

kserve-ci-e2e-test

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-2hv7k

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-2hv7k to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-5np5b

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-5np5b to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9df8wz4

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9df8wz4 to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb46pms2

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb46pms2 to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0 to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

scheduler-custom-template-test-kserve-5bc7bd6864-jbf8p

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-jbf8p to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

scheduler-inline-config-test-kserve-router-scheduler-d9ddfsqtmv

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfsqtmv to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

auth-enabled-test-kserve-67f6dd7f6f-p6frc

Scheduled

Successfully assigned kserve-ci-e2e-test/auth-enabled-test-kserve-67f6dd7f6f-p6frc to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879cvnz8

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879cvnz8 to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1 to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schekdmw4

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schekdmw4 to ip-10-0-142-172.ec2.internal

kserve-ci-e2e-test

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-schezjgtq

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-schezjgtq to ip-10-0-142-172.ec2.internal

kserve-ci-e2e-test

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schebbsmf

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schebbsmf to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-test-kserve-86c9c75755-7p6kx

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-test-kserve-86c9c75755-7p6kx to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-hwz6g

Scheduled

Successfully assigned kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-hwz6g to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

stop-feature-test-kserve-router-scheduler-7cb57d7b84-rn8hp

Scheduled

Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb57d7b84-rn8hp to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schen2sbt

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schen2sbt to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

router-gateway-1-openshift-default-54c47cdfc7-gb4wz

Scheduled

Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-gb4wz to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

precise-prefix-cache-test-kserve-595cd9479-68wpf

Scheduled

Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-595cd9479-68wpf to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

precise-prefix-cache-test-kserve-595cd9479-9fl82

Scheduled

Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-595cd9479-9fl82 to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

auth-enabled-test-kserve-router-scheduler-857b4f747c-7k9fn

Scheduled

Successfully assigned kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-857b4f747c-7k9fn to ip-10-0-128-142.ec2.internal

kuadrant-system

kuadrant-operator-controller-manager-bb7ddb7d7-tldlw

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-tldlw to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-66b95f57dfnrhp7

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-66b95f57dfnrhp7 to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

precise-prefix-cache-test-kserve-router-scheduler-77df687bw4gt6

Scheduled

Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-77df687bw4gt6 to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5989gpd

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5989gpd to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

prestop-hook-test-kserve-5b7d7d79ff-52bt2

Scheduled

Successfully assigned kserve-ci-e2e-test/prestop-hook-test-kserve-5b7d7d79ff-52bt2 to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

router-gateway-1-openshift-default-54c47cdfc7-rxfpz

Scheduled

Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-rxfpz to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-59596f6d4fsgc99

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-59596f6d4fsgc99 to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schezdkxl

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schezdkxl to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

auth-disabled-test-kserve-router-scheduler-7575d494fd-whhqz

Scheduled

Successfully assigned kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-7575d494fd-whhqz to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

e2e-pvc-model-download-bsd5v

Scheduled

Successfully assigned kserve-ci-e2e-test/e2e-pvc-model-download-bsd5v to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

scheduler-v06-threshold-migration-test-kserve-router-sched7z678

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched7z678 to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

router-gateway-1-openshift-default-799f46c59b-44gcv

Scheduled

Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-44gcv to ip-10-0-142-172.ec2.internal

kserve-ci-e2e-test

router-gateway-1-openshift-default-799f46c59b-6gdsh

Scheduled

Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-6gdsh to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-3c960099-kserve-6d69fc44d8842wn

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-6d69fc44d8842wn to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

scheduler-v06-threshold-migration-test-kserve-5994569d5b-cn6jx

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-cn6jx to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schepcckr

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schepcckr to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schex2hpd

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schex2hpd to ip-10-0-142-172.ec2.internal

kserve-ci-e2e-test

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schehqcg7

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schehqcg7 to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche58jw2

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche58jw2 to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche82tj8

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche82tj8 to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schepcpc7

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schepcpc7 to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schegd925

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schegd925 to ip-10-0-142-172.ec2.internal

kserve-ci-e2e-test

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep77sg

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep77sg to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheckgmc

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheckgmc to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-6c94dd65d4-jd986

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-6c94dd65d4-jd986 to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

lora-multiple-adapters-test-kserve-7c8db66c87-pzn59

Scheduled

Successfully assigned kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-7c8db66c87-pzn59 to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

auth-disabled-test-kserve-84ff994f99-sl6v4

Scheduled

Successfully assigned kserve-ci-e2e-test/auth-disabled-test-kserve-84ff994f99-sl6v4 to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

auth-invalid-token-test-kserve-69f89b8bf7-whpjz

Scheduled

Successfully assigned kserve-ci-e2e-test/auth-invalid-token-test-kserve-69f89b8bf7-whpjz to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

scheduler-v06-pd-migration-test-kserve-router-scheduler-595ls74

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-595ls74 to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-dqd46

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-dqd46 to ip-10-0-142-172.ec2.internal

kserve-ci-e2e-test

scheduler-v06-pd-migration-test-kserve-5c87756974-tqwvx

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-tqwvx to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

stop-feature-test-kserve-7c7845f79f-knxs6

Scheduled

Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-7c7845f79f-knxs6 to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

router-gateway-1-openshift-default-799f46c59b-dw428

Scheduled

Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-dw428 to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

stop-feature-test-kserve-7c7845f79f-tfvwg

Scheduled

Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-7c7845f79f-tfvwg to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0 to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4wjhf9

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4wjhf9 to ip-10-0-142-172.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-pd-test-kserve-router-scheduler-74789fhwf6

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74789fhwf6 to ip-10-0-130-182.ec2.internal

kuadrant-system

limitador-operator-controller-manager-6f9f468797-kh4pg

Scheduled

Successfully assigned kuadrant-system/limitador-operator-controller-manager-6f9f468797-kh4pg to ip-10-0-142-172.ec2.internal

openshift-ingress

openshift-ai-inference-openshift-default-655f7c58b4-mbx7z

Scheduled

Successfully assigned openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-mbx7z to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

auth-invalid-token-test-kserve-router-scheduler-5478bf674ccw6tc

Scheduled

Successfully assigned kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-5478bf674ccw6tc to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

router-gateway-1-openshift-default-799f46c59b-z6cd2

Scheduled

Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-z6cd2 to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

scheduler-configmap-ref-test-kserve-router-scheduler-68f87xj26f

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87xj26f to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

stop-feature-test-kserve-router-scheduler-7cb57d7b84-rhxsb

Scheduled

Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb57d7b84-rhxsb to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

scheduler-configmap-ref-test-kserve-744bf59fbd-g7knk

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-g7knk to ip-10-0-142-172.ec2.internal

openshift-ingress

istiod-openshift-gateway-94bb8fbfd-5kc7r

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5kc7r to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

router-with-refs-test-kserve-router-scheduler-85b8956674-9vvqt

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-85b8956674-9vvqt to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

router-with-refs-test-kserve-697958d7-hhdqd

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-test-kserve-697958d7-hhdqd to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

router-with-refs-pd-test-kserve-router-scheduler-68dbc8995xw9h6

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-68dbc8995xw9h6 to ip-10-0-130-182.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-pd-test-kserve-prefill-94b6f6947-rx456

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-94b6f6947-rx456 to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

router-gateway-2-openshift-default-54c789bdc6-tppn4

Scheduled

Successfully assigned kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-tppn4 to ip-10-0-142-172.ec2.internal

kserve-ci-e2e-test

router-gateway-2-openshift-default-684dc9b4b9-h2x2d

Scheduled

Successfully assigned kserve-ci-e2e-test/router-gateway-2-openshift-default-684dc9b4b9-h2x2d to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

router-with-refs-pd-test-kserve-57d4965598-p2stc

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-57d4965598-p2stc to ip-10-0-128-142.ec2.internal

kserve-ci-e2e-test

router-with-refs-pd-test-kserve-prefill-5c9fc7cc79-2ggqn

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c9fc7cc79-2ggqn to ip-10-0-128-142.ec2.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-5658d689b9 from 0 to 1

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer
(x11)

openshift-ingress

replicaset-controller

router-default-5658d689b9

FailedCreate

Error creating: pods "router-default-5658d689b9-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x16)

openshift-ingress

replicaset-controller

router-default-5658d689b9

FailedCreate

Error creating: pods "router-default-5658d689b9-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts

openshift-ingress

replicaset-controller

router-default-5658d689b9

SuccessfulCreate

Created pod: router-default-5658d689b9-4zvzx
(x6)

openshift-ingress

kubelet

router-default-5658d689b9-4zvzx

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x6)

openshift-ingress

kubelet

router-default-5658d689b9-4zvzx

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt

openshift-ingress

kubelet

router-default-5658d689b9-4zvzx

Created

Created container: router

openshift-ingress

kubelet

router-default-5658d689b9-4zvzx

Started

Started container router

openshift-ingress

kubelet

router-default-5658d689b9-4zvzx

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:32d6df0cb6282bc5ba571a8de322fe598afdf22e817f5a7dc92db3274dbdb9a1" already present on machine

openshift-ingress

multus

router-default-5658d689b9-4zvzx

AddedInterface

Add eth0 [10.132.0.16/23] from ovn-kubernetes

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-94bb8fbfd from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-94bb8fbfd

SuccessfulCreate

Created pod: istiod-openshift-gateway-94bb8fbfd-5kc7r

openshift-ingress

kubelet

istiod-openshift-gateway-94bb8fbfd-5kc7r

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0850242436e88f7d82f0f2126de064c7e0f09844f31d8ff0f53dc8d3908075d9"

openshift-ingress

multus

istiod-openshift-gateway-94bb8fbfd-5kc7r

AddedInterface

Add eth0 [10.133.0.29/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-94bb8fbfd-5kc7r

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0850242436e88f7d82f0f2126de064c7e0f09844f31d8ff0f53dc8d3908075d9" in 3.597s (3.597s including waiting). Image size: 181034515 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-94bb8fbfd-5kc7r

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-94bb8fbfd-5kc7r

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-94bb8fbfd-5kc7r

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-94bb8fbfd-5kc7r

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

service-controller

openshift-ai-inference-openshift-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

deployment-controller

openshift-ai-inference-openshift-default

ScalingReplicaSet

Scaled up replica set openshift-ai-inference-openshift-default-655f7c58b4 from 0 to 1

openshift-ingress

multus

openshift-ai-inference-openshift-default-655f7c58b4-mbx7z

AddedInterface

Add eth0 [10.132.0.31/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

openshift-ai-inference

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway openshift-ai-inference

openshift-ingress

replicaset-controller

openshift-ai-inference-openshift-default-655f7c58b4

SuccessfulCreate

Created pod: openshift-ai-inference-openshift-default-655f7c58b4-mbx7z

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-655f7c58b4-mbx7z

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8"

openshift-ingress

service-controller

openshift-ai-inference-openshift-default

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-655f7c58b4-mbx7z

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" in 3.131s (3.131s including waiting). Image size: 181251556 bytes.

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.0

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.0

RequirementsUnknown

requirements not yet checked
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.0

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.0

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

deployment-controller

dns-operator-controller-manager

ScalingReplicaSet

Scaled up replica set dns-operator-controller-manager-65b49595d7 from 0 to 1

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.1

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.0

InstallWaiting

installing: waiting for deployment dns-operator-controller-manager to become ready: deployment "dns-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

kubelet

dns-operator-controller-manager-65b49595d7-dx5b4

Pulling

Pulling image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:ba46f54446ecd3ed1a3d877dcbb7359ff8f2d918827f3411a5cb0781915a65d5"

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.1

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

replicaset-controller

dns-operator-controller-manager-65b49595d7

SuccessfulCreate

Created pod: dns-operator-controller-manager-65b49595d7-dx5b4

kuadrant-system

multus

dns-operator-controller-manager-65b49595d7-dx5b4

AddedInterface

Add eth0 [10.133.0.34/23] from ovn-kubernetes

kuadrant-system

kubelet

dns-operator-controller-manager-65b49595d7-dx5b4

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:ba46f54446ecd3ed1a3d877dcbb7359ff8f2d918827f3411a5cb0781915a65d5" in 1.965s (1.965s including waiting). Image size: 190472853 bytes.

kuadrant-system

dns-operator-controller-manager-65b49595d7-dx5b4_418cd74a-7f5e-4164-a60b-e2569c21eac3

a3f98d6c.kuadrant.io

LeaderElection

dns-operator-controller-manager-65b49595d7-dx5b4_418cd74a-7f5e-4164-a60b-e2569c21eac3 became leader

kuadrant-system

kubelet

dns-operator-controller-manager-65b49595d7-dx5b4

Started

Started container manager

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.1

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

kubelet

dns-operator-controller-manager-65b49595d7-dx5b4

Created

Created container: manager

kuadrant-system

replicaset-controller

authorino-operator-6d85f6564

SuccessfulCreate

Created pod: authorino-operator-6d85f6564-hp27q

kuadrant-system

kubelet

authorino-operator-6d85f6564-hp27q

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:b2cc2c0f0729dad98f970830816e300c0b961ba42491f362295512d886384687"

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.1

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.1

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.1

InstallWaiting

installing: waiting for deployment authorino-operator to become ready: deployment "authorino-operator" not available: Deployment does not have minimum availability.

kuadrant-system

deployment-controller

authorino-operator

ScalingReplicaSet

Scaled up replica set authorino-operator-6d85f6564 from 0 to 1

kuadrant-system

multus

authorino-operator-6d85f6564-hp27q

AddedInterface

Add eth0 [10.133.0.36/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.1

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

kubelet

authorino-operator-6d85f6564-hp27q

Started

Started container manager

kuadrant-system

kubelet

authorino-operator-6d85f6564-hp27q

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:b2cc2c0f0729dad98f970830816e300c0b961ba42491f362295512d886384687" in 2.142s (2.142s including waiting). Image size: 168639631 bytes.

kuadrant-system

kubelet

authorino-operator-6d85f6564-hp27q

Created

Created container: manager

kuadrant-system

authorino-operator-6d85f6564-hp27q_57781789-9b58-4da5-8d3e-dc34c399dd23

aac3a15d.authorino.kuadrant.io

LeaderElection

authorino-operator-6d85f6564-hp27q_57781789-9b58-4da5-8d3e-dc34c399dd23 became leader

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.1

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.1

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.1

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.0

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.0

RequirementsUnknown

requirements not yet checked

kuadrant-system

multus

kuadrant-operator-controller-manager-bb7ddb7d7-tldlw

AddedInterface

Add eth0 [10.133.0.37/23] from ovn-kubernetes

kuadrant-system

kubelet

kuadrant-operator-controller-manager-bb7ddb7d7-tldlw

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:0b77e4a874b620095a0300877aea53e78559a601ce62b194537ed21c2743bbf3"

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.0

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-bb7ddb7d7

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-bb7ddb7d7-tldlw

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-bb7ddb7d7 from 0 to 1

kuadrant-system

kubelet

limitador-operator-controller-manager-6f9f468797-kh4pg

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:183bad2a6f5872aea38f54ad507f7e005d6facb02b18769e0407c82799f96a3e"

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.0

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

multus

limitador-operator-controller-manager-6f9f468797-kh4pg

AddedInterface

Add eth0 [10.134.0.17/23] from ovn-kubernetes

kuadrant-system

replicaset-controller

limitador-operator-controller-manager-6f9f468797

SuccessfulCreate

Created pod: limitador-operator-controller-manager-6f9f468797-kh4pg

kuadrant-system

deployment-controller

limitador-operator-controller-manager

ScalingReplicaSet

Scaled up replica set limitador-operator-controller-manager-6f9f468797 from 0 to 1

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.0

InstallWaiting

installing: waiting for deployment limitador-operator-controller-manager to become ready: deployment "limitador-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

kubelet

limitador-operator-controller-manager-6f9f468797-kh4pg

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:183bad2a6f5872aea38f54ad507f7e005d6facb02b18769e0407c82799f96a3e" in 2.115s (2.115s including waiting). Image size: 178538972 bytes.

kuadrant-system

kubelet

limitador-operator-controller-manager-6f9f468797-kh4pg

Created

Created container: manager

kuadrant-system

kubelet

limitador-operator-controller-manager-6f9f468797-kh4pg

Started

Started container manager

kuadrant-system

kubelet

kuadrant-operator-controller-manager-bb7ddb7d7-tldlw

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:0b77e4a874b620095a0300877aea53e78559a601ce62b194537ed21c2743bbf3" in 3.707s (3.707s including waiting). Image size: 425661757 bytes.

kuadrant-system

kubelet

kuadrant-operator-controller-manager-bb7ddb7d7-tldlw

Started

Started container manager

kuadrant-system

kubelet

kuadrant-operator-controller-manager-bb7ddb7d7-tldlw

Created

Created container: manager

kuadrant-system

limitador-operator-controller-manager-6f9f468797-kh4pg_0ff071c4-1141-4c7b-8e33-12272890b942

3745a16e.kuadrant.io

LeaderElection

limitador-operator-controller-manager-6f9f468797-kh4pg_0ff071c4-1141-4c7b-8e33-12272890b942 became leader

kuadrant-system

kuadrant-operator-controller-manager-bb7ddb7d7-tldlw_6c02e1ce-f124-4f94-af3b-729784f80e67

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-bb7ddb7d7-tldlw_6c02e1ce-f124-4f94-af3b-729784f80e67 became leader

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.0

InstallSucceeded

install strategy completed with no errors

kuadrant-system

deployment-controller

kuadrant-console-plugin

ScalingReplicaSet

Scaled up replica set kuadrant-console-plugin-85fd4b8b9d from 0 to 1

kuadrant-system

replicaset-controller

kuadrant-console-plugin-85fd4b8b9d

SuccessfulCreate

Created pod: kuadrant-console-plugin-85fd4b8b9d-28qld

kuadrant-system

kubelet

kuadrant-console-plugin-85fd4b8b9d-28qld

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:6e69f41d3d70b0afa7691a5d2400065d3bcdec2ea82830cf2992b8e69ba06e2a"

kuadrant-system

multus

kuadrant-console-plugin-85fd4b8b9d-28qld

AddedInterface

Add eth0 [10.132.0.33/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.1

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kubelet

kuadrant-console-plugin-85fd4b8b9d-28qld

Created

Created container: kuadrant-console-plugin

kuadrant-system

kubelet

kuadrant-console-plugin-85fd4b8b9d-28qld

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:6e69f41d3d70b0afa7691a5d2400065d3bcdec2ea82830cf2992b8e69ba06e2a" in 5.987s (5.987s including waiting). Image size: 342502139 bytes.

kuadrant-system

kubelet

kuadrant-console-plugin-85fd4b8b9d-28qld

Started

Started container kuadrant-console-plugin

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.0

InstallSucceeded

install strategy completed with no errors

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.1

InstallSucceeded

install strategy completed with no errors

kuadrant-system

multus

limitador-limitador-ff68ff5cb-lx9ww

AddedInterface

Add eth0 [10.133.0.38/23] from ovn-kubernetes

kuadrant-system

replicaset-controller

limitador-limitador-69574b596d

SuccessfulCreate

Created pod: limitador-limitador-69574b596d-gxr6q

kuadrant-system

kubelet

limitador-limitador-ff68ff5cb-lx9ww

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:0b02c000d9a175235223ec5916345a1ccedeedc25c87cc0638e8554e6c318474"

kuadrant-system

replicaset-controller

limitador-limitador-ff68ff5cb

SuccessfulCreate

Created pod: limitador-limitador-ff68ff5cb-lx9ww

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-69574b596d from 0 to 1

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-ff68ff5cb from 0 to 1

kuadrant-system

multus

authorino-b6c89bb85-6smtk

AddedInterface

Add eth0 [10.132.0.34/23] from ovn-kubernetes

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-8df697c9f from 0 to 1

kuadrant-system

replicaset-controller

authorino-b6c89bb85

SuccessfulCreate

Created pod: authorino-b6c89bb85-6smtk

kuadrant-system

replicaset-controller

authorino-8df697c9f

SuccessfulCreate

Created pod: authorino-8df697c9f-zjlcz

kuadrant-system

multus

limitador-limitador-69574b596d-gxr6q

AddedInterface

Add eth0 [10.134.0.18/23] from ovn-kubernetes

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-b6c89bb85 from 0 to 1

kuadrant-system

kubelet

limitador-limitador-69574b596d-gxr6q

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:0b02c000d9a175235223ec5916345a1ccedeedc25c87cc0638e8554e6c318474"

kuadrant-system

kubelet

authorino-8df697c9f-zjlcz

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a"

kuadrant-system

multus

authorino-8df697c9f-zjlcz

AddedInterface

Add eth0 [10.133.0.39/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-b6c89bb85-6smtk

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a"

kuadrant-system

kubelet

limitador-limitador-69574b596d-gxr6q

Created

Created container: limitador

kuadrant-system

kubelet

limitador-limitador-69574b596d-gxr6q

Started

Started container limitador

kuadrant-system

kubelet

limitador-limitador-69574b596d-gxr6q

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:0b02c000d9a175235223ec5916345a1ccedeedc25c87cc0638e8554e6c318474" in 3.495s (3.495s including waiting). Image size: 250499765 bytes.

kuadrant-system

kubelet

authorino-b6c89bb85-6smtk

Started

Started container authorino

kuadrant-system

kubelet

authorino-b6c89bb85-6smtk

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a" in 3.82s (3.82s including waiting). Image size: 202706732 bytes.

kuadrant-system

kubelet

authorino-b6c89bb85-6smtk

Created

Created container: authorino

kuadrant-system

kubelet

authorino-b6c89bb85-6smtk

Killing

Stopping container authorino

kuadrant-system

replicaset-controller

authorino-b6c89bb85

SuccessfulDelete

Deleted pod: authorino-b6c89bb85-6smtk

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-b6c89bb85 from 1 to 0

kuadrant-system

kubelet

authorino-8df697c9f-zjlcz

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a" in 5.192s (5.192s including waiting). Image size: 202706732 bytes.

kuadrant-system

kubelet

authorino-8df697c9f-zjlcz

Started

Started container authorino

kuadrant-system

kubelet

limitador-limitador-ff68ff5cb-lx9ww

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:0b02c000d9a175235223ec5916345a1ccedeedc25c87cc0638e8554e6c318474" in 6.325s (6.325s including waiting). Image size: 250499765 bytes.

kuadrant-system

kubelet

authorino-8df697c9f-zjlcz

Created

Created container: authorino

kuadrant-system

kubelet

limitador-limitador-ff68ff5cb-lx9ww

Created

Created container: limitador

kuadrant-system

kubelet

limitador-limitador-ff68ff5cb-lx9ww

Started

Started container limitador

kuadrant-system

replicaset-controller

limitador-limitador-ff68ff5cb

SuccessfulDelete

Deleted pod: limitador-limitador-ff68ff5cb-lx9ww

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled down replica set limitador-limitador-ff68ff5cb from 1 to 0

kuadrant-system

kubelet

limitador-limitador-ff68ff5cb-lx9ww

Killing

Stopping container limitador

kuadrant-system

kubelet

authorino-7c77957575-lkzck

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a"

kuadrant-system

replicaset-controller

authorino-7c77957575

SuccessfulCreate

Created pod: authorino-7c77957575-lkzck

kuadrant-system

multus

authorino-7c77957575-lkzck

AddedInterface

Add eth0 [10.133.0.40/23] from ovn-kubernetes

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-7c77957575 from 0 to 1

kuadrant-system

replicaset-controller

authorino-8df697c9f

SuccessfulDelete

Deleted pod: authorino-8df697c9f-zjlcz

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-8df697c9f from 1 to 0

kuadrant-system

kubelet

authorino-7c77957575-lkzck

Started

Started container authorino

kuadrant-system

kubelet

authorino-7c77957575-lkzck

Created

Created container: authorino

kuadrant-system

kubelet

authorino-7c77957575-lkzck

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a" in 329ms (329ms including waiting). Image size: 202706732 bytes.

kuadrant-system

kubelet

authorino-8df697c9f-zjlcz

Killing

Stopping container authorino

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-6gdsh

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8"

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-54c47cdfc7

SuccessfulCreate

Created pod: router-gateway-1-openshift-default-54c47cdfc7-gb4wz

kserve-ci-e2e-test

multus

router-gateway-1-openshift-default-799f46c59b-6gdsh

AddedInterface

Add eth0 [10.133.0.44/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-799f46c59b

SuccessfulCreate

Created pod: router-gateway-1-openshift-default-799f46c59b-6gdsh

kserve-ci-e2e-test

service-controller

router-gateway-1-openshift-default

EnsuringLoadBalancer

Ensuring load balancer

kserve-ci-e2e-test

multus

router-gateway-1-openshift-default-54c47cdfc7-gb4wz

AddedInterface

Add eth0 [10.132.0.36/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-gb4wz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-gb4wz

Started

Started container istio-proxy

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-gb4wz

Created

Created container: istio-proxy

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-6gdsh

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" in 2.945s (2.945s including waiting). Image size: 181251556 bytes.

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-6gdsh

Created

Created container: istio-proxy

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-6gdsh

Started

Started container istio-proxy

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-799f46c59b

SuccessfulDelete

Deleted pod: router-gateway-1-openshift-default-799f46c59b-6gdsh

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-6gdsh

Killing

Stopping container istio-proxy

kserve-ci-e2e-test

service-controller

router-gateway-1-openshift-default

EnsuredLoadBalancer

Ensured load balancer

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.Service kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-workload-svc

kserve-ci-e2e-test

replicaset-controller

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc5

SuccessfulCreate

Created pod: gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc526tch

kserve-ci-e2e-test

deployment-controller

gw-section-name-router-with-gat-2f0a622e-kserve

ScalingReplicaSet

Scaled up replica set gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc5 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.Secret kserve-ci-e2e-test/gw-sec0c69dceeb48768325d1a53a749e65786-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.Deployment kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.HTTPRoute kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.DestinationRule kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-workload-svc

kserve-ci-e2e-test

multus

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc526tch

AddedInterface

Add eth0 [10.132.0.37/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc526tch

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc526tch

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

UpdateFailed

Failed to update status for LLMInferenceService "gw-section-name-router-with-gat-2f0a622e": Operation cannot be fulfilled on llminferenceservices.serving.kserve.io "gw-section-name-router-with-gat-2f0a622e": the object has been modified; please apply your changes to the latest version and try again

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc526tch

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.383s (1.383s including waiting). Image size: 98346788 bytes.

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc526tch

Created

Created container: main

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-799f46c59b

SuccessfulCreate

Created pod: router-gateway-1-openshift-default-799f46c59b-dw428

kserve-ci-e2e-test

multus

router-gateway-1-openshift-default-799f46c59b-dw428

AddedInterface

Add eth0 [10.133.0.45/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

router-gateway-1-openshift-default

ScalingReplicaSet

Scaled down replica set router-gateway-1-openshift-default-54c47cdfc7 from 1 to 0

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-gb4wz

Killing

Stopping container istio-proxy

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-54c47cdfc7

SuccessfulDelete

Deleted pod: router-gateway-1-openshift-default-54c47cdfc7-gb4wz
(x13)

kserve-ci-e2e-test

OpenDataHubModelController

precise-prefix-cache-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy precise-prefix-cache-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/precise-prefix-cache-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "precise-prefix-cache-test-kserve-route" not found

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-gb4wz

Unhealthy

Readiness probe failed: Get "http://10.132.0.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-dw428

Started

Started container istio-proxy
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-dw428

Created

Created container: istio-proxy
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-dw428

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Service kserve-ci-e2e-test/precise-prefix-cache-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Deployment kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler

kserve-ci-e2e-test

replicaset-controller

precise-prefix-cache-test-kserve-router-scheduler-77df687b47

SuccessfulCreate

Created pod: precise-prefix-cache-test-kserve-router-scheduler-77df687bw4gt6

kserve-ci-e2e-test

multus

precise-prefix-cache-test-kserve-595cd9479-9fl82

AddedInterface

Add eth0 [10.133.0.46/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

precise-prefix-cache-test-kserve-595cd9479

SuccessfulCreate

Created pod: precise-prefix-cache-test-kserve-595cd9479-68wpf

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-595cd9479-9fl82

Pulling

Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8"

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-595cd9479-68wpf

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-595cd9479-68wpf

Created

Created container: storage-initializer

kserve-ci-e2e-test

deployment-controller

precise-prefix-cache-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set precise-prefix-cache-test-kserve-router-scheduler-77df687b47 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

precise-prefix-cache-test-kserve-595cd9479

SuccessfulCreate

Created pod: precise-prefix-cache-test-kserve-595cd9479-9fl82

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/precise-prefix-cache-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Role kserve-ci-e2e-test/precise-prefix-cache-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-precise-prefix-cache-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-595cd9479-68wpf

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/precise-prefix-cache-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Service kserve-ci-e2e-test/precise-prefix-cache-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Deployment kserve-ci-e2e-test/precise-prefix-cache-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Secret kserve-ci-e2e-test/precise-prefix-cache-test-kserve-self-signed-certs

kserve-ci-e2e-test

multus

precise-prefix-cache-test-kserve-595cd9479-68wpf

AddedInterface

Add eth0 [10.132.0.38/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

precise-prefix-cache-test-kserve

ScalingReplicaSet

Scaled up replica set precise-prefix-cache-test-kserve-595cd9479 from 0 to 2

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-77df687bw4gt6

Created

Created container: storage-initializer

kserve-ci-e2e-test

multus

precise-prefix-cache-test-kserve-router-scheduler-77df687bw4gt6

AddedInterface

Add eth0 [10.132.0.39/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-77df687bw4gt6

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-77df687bw4gt6

Started

Started container storage-initializer
(x5)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-dw428

Unhealthy

Startup probe failed: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-77df687bw4gt6

Pulling

Pulling image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2"

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc526tch

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-77df687bw4gt6

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" in 1.655s (1.655s including waiting). Image size: 75073927 bytes.

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-77df687bw4gt6

Pulling

Pulling image "ghcr.io/llm-d/llm-d-uds-tokenizer:vllm-v0.19.1"

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-595cd9479-9fl82

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-595cd9479-9fl82

Pulled

Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" in 4.624s (4.624s including waiting). Image size: 301879267 bytes.

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-595cd9479-9fl82

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-595cd9479-68wpf

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-595cd9479-68wpf

Created

Created container: main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-595cd9479-68wpf

Started

Started container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-595cd9479-9fl82

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-595cd9479-9fl82

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.035s (1.035s including waiting). Image size: 98346788 bytes.

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-595cd9479-9fl82

Started

Started container main

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-655f7c58b4-mbx7z

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
(x2)

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-655f7c58b4-mbx7z

Created

Created container: istio-proxy

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-595cd9479-9fl82

Created

Created container: main
(x2)

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-655f7c58b4-mbx7z

Started

Started container istio-proxy

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.DestinationRule kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-workload-svc

kserve-ci-e2e-test

deployment-controller

gw-section-name-router-with-gat-f1d92d0f-kserve

ScalingReplicaSet

Scaled up replica set gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b from 0 to 1

kserve-ci-e2e-test

replicaset-controller

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b

SuccessfulCreate

Created pod: gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwm7dq

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.HTTPRoute kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.Service kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.Deployment kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.Secret kserve-ci-e2e-test/gw-sec2774c263d49959f50d9eebc552e13bf9-kserve-self-signed-certs

kserve-ci-e2e-test

multus

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwm7dq

AddedInterface

Add eth0 [10.134.0.19/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwm7dq

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwm7dq

Created

Created container: main
(x5)

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-655f7c58b4-mbx7z

ProbeError

Startup probe error: Get "http://10.132.0.31:15021/healthz/ready": dial tcp 10.132.0.31:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-655f7c58b4-mbx7z

Unhealthy

Startup probe failed: Get "http://10.132.0.31:15021/healthz/ready": dial tcp 10.132.0.31:15021: connect: connection refused

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc526tch

Unhealthy

Readiness probe failed: Get "https://10.132.0.37:8000/health": dial tcp 10.132.0.37:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwm7dq

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

UpdateFailed

Failed to update status for LLMInferenceService "gw-section-name-router-with-gat-f1d92d0f": Operation cannot be fulfilled on llminferenceservices.serving.kserve.io "gw-section-name-router-with-gat-f1d92d0f": the object has been modified; please apply your changes to the latest version and try again

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwm7dq

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.017s (1.017s including waiting). Image size: 98346788 bytes.
(x2)

kserve-ci-e2e-test

deployment-controller

router-gateway-1-openshift-default

ScalingReplicaSet

Scaled up replica set router-gateway-1-openshift-default-54c47cdfc7 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-54c47cdfc7

SuccessfulCreate

Created pod: router-gateway-1-openshift-default-54c47cdfc7-rxfpz

kserve-ci-e2e-test

multus

router-gateway-1-openshift-default-54c47cdfc7-rxfpz

AddedInterface

Add eth0 [10.133.0.47/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-dw428

Killing

Stopping container istio-proxy

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-799f46c59b

SuccessfulDelete

Deleted pod: router-gateway-1-openshift-default-799f46c59b-dw428

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Updated

Updated v1.Secret kserve-ci-e2e-test/precise-prefix-cache-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/precise-prefix-cache-test-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/precise-prefix-cache-test-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwm7dq

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-77df687bw4gt6

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-77df687bw4gt6

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-77df687bw4gt6

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-uds-tokenizer:vllm-v0.19.1" in 33.295s (33.295s including waiting). Image size: 2989890188 bytes.
(x2)

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-77df687bw4gt6

Started

Started container main
(x2)

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-77df687bw4gt6

Created

Created container: main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-77df687bw4gt6

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwm7dq

Unhealthy

Readiness probe failed: Get "https://10.134.0.19:8000/health": dial tcp 10.134.0.19:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.Secret kserve-ci-e2e-test/auth-enabled-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.Service kserve-ci-e2e-test/auth-enabled-test-kserve-workload-svc

kserve-ci-e2e-test

deployment-controller

auth-enabled-test-kserve

ScalingReplicaSet

Scaled up replica set auth-enabled-test-kserve-67f6dd7f6f from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.Deployment kserve-ci-e2e-test/auth-enabled-test-kserve

kserve-ci-e2e-test

replicaset-controller

auth-enabled-test-kserve-67f6dd7f6f

SuccessfulCreate

Created pod: auth-enabled-test-kserve-67f6dd7f6f-p6frc

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.Deployment kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.Service kserve-ci-e2e-test/auth-enabled-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/auth-enabled-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-enabled-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-67f6dd7f6f-p6frc

Created

Created container: storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/auth-enabled-test-epp-rb

kserve-ci-e2e-test

multus

auth-enabled-test-kserve-67f6dd7f6f-p6frc

AddedInterface

Add eth0 [10.133.0.48/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

auth-enabled-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set auth-enabled-test-kserve-router-scheduler-857b4f747c from 0 to 1

kserve-ci-e2e-test

replicaset-controller

auth-enabled-test-kserve-router-scheduler-857b4f747c

SuccessfulCreate

Created pod: auth-enabled-test-kserve-router-scheduler-857b4f747c-7k9fn

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-857b4f747c-7k9fn

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-67f6dd7f6f-p6frc

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-857b4f747c-7k9fn

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-857b4f747c-7k9fn

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

multus

auth-enabled-test-kserve-router-scheduler-857b4f747c-7k9fn

AddedInterface

Add eth0 [10.133.0.49/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.Role kserve-ci-e2e-test/auth-enabled-test-epp-role

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-67f6dd7f6f-p6frc

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-857b4f747c-7k9fn

Pulling

Pulling image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2"

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-857b4f747c-7k9fn

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" in 1.645s (1.645s including waiting). Image size: 75073927 bytes.

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-77df687bw4gt6

Unhealthy

Liveness probe failed: timeout: failed to connect service "10.132.0.39:9003" within 1s: context deadline exceeded

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-857b4f747c-7k9fn

Created

Created container: main

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-857b4f747c-7k9fn

Started

Started container main

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-67f6dd7f6f-p6frc

Pulling

Pulling image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0"

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/auth-enabled-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Updated

Updated v1.Secret kserve-ci-e2e-test/auth-enabled-test-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/auth-enabled-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

LLMInferenceServiceReady

LLMInferenceService [precise-prefix-cache-test] is Ready

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-595cd9479-9fl82

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-77df687bw4gt6

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-77df687bw4gt6

Killing

Stopping container tokenizer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-77df687bw4gt6

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-precise-prefix-cache-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-595cd9479-68wpf

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-67f6dd7f6f-p6frc

Created

Created container: main

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-67f6dd7f6f-p6frc

Started

Started container main

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-67f6dd7f6f-p6frc

Pulled

Successfully pulled image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" in 28.715s (28.715s including waiting). Image size: 3531177328 bytes.

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-595cd9479-68wpf

Unhealthy

Readiness probe failed: Get "https://10.132.0.38:8000/health": dial tcp 10.132.0.38:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-595cd9479-9fl82

Unhealthy

Readiness probe failed: Get "https://10.133.0.46:8000/health": dial tcp 10.133.0.46:8000: connect: connection refused
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-router-managed-test-llm-e45d1f79

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-router-managed-test-llm-e45d1f79-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-router-managed-test-llm-e45d1f79-kserve-route" not found

kserve-ci-e2e-test

deployment-controller

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheduler-5dd88bfbb7 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheduler

kserve-ci-e2e-test

replicaset-controller

llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879

SuccessfulCreate

Created pod: llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879cvnz8

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879cvnz8

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.Secret kserve-ci-e2e-test/llmisv5c7e67b6c51568d1d6d13829a9337f2a-kserve-self-signed-certs

kserve-ci-e2e-test

replicaset-controller

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheduler-5dd88bfbb7

SuccessfulCreate

Created pod: llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schex2hpd

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879cvnz8

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve

kserve-ci-e2e-test

deployment-controller

llmisvc-router-managed-test-llm-e45d1f79-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-testef4d2875be14b30dc1561ed84d0d4bde-llmisvc-rout

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-service

kserve-ci-e2e-test

multus

llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879cvnz8

AddedInterface

Add eth0 [10.132.0.40/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-role

kserve-ci-e2e-test

kubelet

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schex2hpd

Pulling

Pulling image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2"

kserve-ci-e2e-test

multus

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schex2hpd

AddedInterface

Add eth0 [10.134.0.20/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879cvnz8

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schex2hpd

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schex2hpd

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" in 1.56s (1.56s including waiting). Image size: 75073927 bytes.

kserve-ci-e2e-test

kubelet

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schex2hpd

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv5c7e67b6c51568d1d6d13829a9337f2a-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

LLMInferenceServiceReady

LLMInferenceService [llmisvc-router-managed-test-llm-e45d1f79] is Ready

kserve-ci-e2e-test

kubelet

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schex2hpd

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-testef4d2875be14b30dc1561ed84d0d4bde-llmisvc-rout

kserve-ci-e2e-test

kubelet

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schex2hpd

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879cvnz8

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879cvnz8

Unhealthy

Readiness probe failed: Get "https://10.132.0.40:8000/health": dial tcp 10.132.0.40:8000: connect: connection refused
(x13)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-router-managed-test-llm-5b1e8f15

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-router-managed-test-llm-5b1e8f15-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-router-managed-test-llm-5b1e8f15-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.Secret kserve-ci-e2e-test/llmisve55ae740357a3a31a27cdb8b66ffe20f-kserve-self-signed-certs

kserve-ci-e2e-test

deployment-controller

llmisvc-router-managed-test-llm-5b1e8f15-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4

SuccessfulCreate

Created pod: llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb46pms2

kserve-ci-e2e-test

kubelet

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheckgmc

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-sa

kserve-ci-e2e-test

multus

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheckgmc

AddedInterface

Add eth0 [10.132.0.42/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheckgmc

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheckgmc

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheduler

kserve-ci-e2e-test

replicaset-controller

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheduler-749449dbc8

SuccessfulCreate

Created pod: llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheckgmc

kserve-ci-e2e-test

deployment-controller

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheduler-749449dbc8 from 0 to 1

kserve-ci-e2e-test

multus

llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb46pms2

AddedInterface

Add eth0 [10.132.0.41/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-service

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb46pms2

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb46pms2

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb46pms2

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test7f54e84970003a6e7372bdbcb574f7ed-llmisvc-rout

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisve55ae740357a3a31a27cdb8b66ffe20f-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-shadow-svc
(x13)

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-67f6dd7f6f-p6frc

Unhealthy

Startup probe failed: Get "https://10.133.0.48:8000/health": dial tcp 10.133.0.48:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

LLMInferenceServiceReady

LLMInferenceService [auth-enabled-test] is Ready

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-67f6dd7f6f-p6frc

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-857b4f747c-7k9fn

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-857b4f747c-7k9fn

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-enabled-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

LLMInferenceServiceReady

LLMInferenceService [llmisvc-router-managed-test-llm-5b1e8f15] is Ready

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-67f6dd7f6f-p6frc

Unhealthy

Readiness probe failed: Get "https://10.133.0.48:8000/health": dial tcp 10.133.0.48:8000: connect: connection refused
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.Secret kserve-ci-e2e-test/llmisvb11a9c9832b99b016bc8f8e0ea095712-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-workload-svc

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58dd

SuccessfulCreate

Created pod: llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddk5j4w

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-testb7025ca4d8a6f8f5b2fd08b5581d2678-llmisvc-mode

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-sa

kserve-ci-e2e-test

deployment-controller

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58dd from 0 to 1

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddk5j4w

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche82tj8

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-rb

kserve-ci-e2e-test

multus

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche82tj8

AddedInterface

Add eth0 [10.133.0.50/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddk5j4w

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddk5j4w

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche82tj8

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-role

kserve-ci-e2e-test

kubelet

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche82tj8

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-service

kserve-ci-e2e-test

multus

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddk5j4w

AddedInterface

Add eth0 [10.134.0.21/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheduler-6d86bd4d9d

SuccessfulCreate

Created pod: llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche82tj8

kserve-ci-e2e-test

deployment-controller

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheduler-6d86bd4d9d from 0 to 1

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-69f89b8bf7-whpjz

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

multus

auth-invalid-token-test-kserve-69f89b8bf7-whpjz

AddedInterface

Add eth0 [10.133.0.51/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.Role kserve-ci-e2e-test/auth-invalid-token-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.Deployment kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.Service kserve-ci-e2e-test/auth-invalid-token-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-invalid-token-test-epp-auth-rb

kserve-ci-e2e-test

replicaset-controller

auth-invalid-token-test-kserve-router-scheduler-5478bf674c

SuccessfulCreate

Created pod: auth-invalid-token-test-kserve-router-scheduler-5478bf674ccw6tc

kserve-ci-e2e-test

replicaset-controller

auth-invalid-token-test-kserve-69f89b8bf7

SuccessfulCreate

Created pod: auth-invalid-token-test-kserve-69f89b8bf7-whpjz

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/auth-invalid-token-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.Service kserve-ci-e2e-test/auth-invalid-token-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.Deployment kserve-ci-e2e-test/auth-invalid-token-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.Secret kserve-ci-e2e-test/auth-invalid-token-test-kserve-self-signed-certs

kserve-ci-e2e-test

deployment-controller

auth-invalid-token-test-kserve

ScalingReplicaSet

Scaled up replica set auth-invalid-token-test-kserve-69f89b8bf7 from 0 to 1

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-69f89b8bf7-whpjz

Created

Created container: storage-initializer

kserve-ci-e2e-test

multus

auth-invalid-token-test-kserve-router-scheduler-5478bf674ccw6tc

AddedInterface

Add eth0 [10.133.0.52/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/auth-invalid-token-test-epp-rb

kserve-ci-e2e-test

deployment-controller

auth-invalid-token-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set auth-invalid-token-test-kserve-router-scheduler-5478bf674c from 0 to 1

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-69f89b8bf7-whpjz

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-5478bf674ccw6tc

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-5478bf674ccw6tc

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-5478bf674ccw6tc

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-5478bf674ccw6tc

Started

Started container main

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-5478bf674ccw6tc

Created

Created container: main

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-5478bf674ccw6tc

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-69f89b8bf7-whpjz

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-69f89b8bf7-whpjz

Started

Started container main

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-69f89b8bf7-whpjz

Created

Created container: main
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisvb11a9c9832b99b016bc8f8e0ea095712-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Updated

Updated v1.Secret kserve-ci-e2e-test/auth-invalid-token-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/auth-invalid-token-test-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/auth-invalid-token-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-qwen2-5-0-5b-rout-a50492e9] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-testb7025ca4d8a6f8f5b2fd08b5581d2678-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddk5j4w

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche82tj8

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche82tj8

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test7f54e84970003a6e7372bdbcb574f7ed-llmisvc-rout

kserve-ci-e2e-test

kubelet

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheckgmc

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheckgmc

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb46pms2

Killing

Stopping container main

kserve-ci-e2e-test

replicaset-controller

llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56

SuccessfulCreate

Created pod: llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-5np5b

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.Secret kserve-ci-e2e-test/llmisvca2d2d7d499abb359505529ebe02c136-kserve-self-signed-certs
(x10)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-router-managed-test-llm-4b931143

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-router-managed-test-llm-4b931143-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-router-managed-test-llm-4b931143-kserve-route" not found

kserve-ci-e2e-test

deployment-controller

llmisvc-router-managed-test-llm-4b931143-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test8ac8e3d2264ccb939eb021b0b835847c-llmisvc-rout

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-rb

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-5np5b

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

multus

llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-5np5b

AddedInterface

Add eth0 [10.132.0.43/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-scheduler-68cc9685d6

SuccessfulCreate

Created pod: llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep77sg

kserve-ci-e2e-test

deployment-controller

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-scheduler-68cc9685d6 from 0 to 1

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-5np5b

Started

Started container main

kserve-ci-e2e-test

multus

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep77sg

AddedInterface

Add eth0 [10.132.0.44/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep77sg

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-5np5b

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep77sg

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-service

kserve-ci-e2e-test

kubelet

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep77sg

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisvca2d2d7d499abb359505529ebe02c136-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

LLMInferenceServiceReady

LLMInferenceService [llmisvc-router-managed-test-llm-4b931143] is Ready
(x12)

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-69f89b8bf7-whpjz

Unhealthy

Startup probe failed: Get "https://10.133.0.51:8000/health": dial tcp 10.133.0.51:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

LLMInferenceServiceReady

LLMInferenceService [auth-invalid-token-test] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-invalid-token-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-5478bf674ccw6tc

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-69f89b8bf7-whpjz

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-5478bf674ccw6tc

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.Deployment kserve-ci-e2e-test/auth-disabled-test-kserve
(x10)

kserve-ci-e2e-test

OpenDataHubModelController

auth-disabled-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy auth-disabled-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/auth-disabled-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "auth-disabled-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.Secret kserve-ci-e2e-test/auth-disabled-test-kserve-self-signed-certs

kserve-ci-e2e-test

multus

auth-disabled-test-kserve-84ff994f99-sl6v4

AddedInterface

Add eth0 [10.133.0.53/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/auth-disabled-test-epp-rb

kserve-ci-e2e-test

replicaset-controller

auth-disabled-test-kserve-84ff994f99

SuccessfulCreate

Created pod: auth-disabled-test-kserve-84ff994f99-sl6v4

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.Role kserve-ci-e2e-test/auth-disabled-test-epp-role

kserve-ci-e2e-test

deployment-controller

auth-disabled-test-kserve

ScalingReplicaSet

Scaled up replica set auth-disabled-test-kserve-84ff994f99 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-disabled-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.Service kserve-ci-e2e-test/auth-disabled-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/auth-disabled-test-epp-sa

kserve-ci-e2e-test

multus

auth-disabled-test-kserve-router-scheduler-7575d494fd-whhqz

AddedInterface

Add eth0 [10.133.0.54/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-84ff994f99-sl6v4

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-router-scheduler-7575d494fd-whhqz

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-84ff994f99-sl6v4

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-84ff994f99-sl6v4

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.Service kserve-ci-e2e-test/auth-disabled-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.Deployment kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-router-scheduler-7575d494fd-whhqz

Created

Created container: storage-initializer

kserve-ci-e2e-test

deployment-controller

auth-disabled-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set auth-disabled-test-kserve-router-scheduler-7575d494fd from 0 to 1

kserve-ci-e2e-test

replicaset-controller

auth-disabled-test-kserve-router-scheduler-7575d494fd

SuccessfulCreate

Created pod: auth-disabled-test-kserve-router-scheduler-7575d494fd-whhqz

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-router-scheduler-7575d494fd-whhqz

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-router-scheduler-7575d494fd-whhqz

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-router-scheduler-7575d494fd-whhqz

Created

Created container: main

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-router-scheduler-7575d494fd-whhqz

Started

Started container main

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-84ff994f99-sl6v4

Created

Created container: main

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-84ff994f99-sl6v4

Started

Started container main

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-84ff994f99-sl6v4

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Updated

Updated v1.Secret kserve-ci-e2e-test/auth-disabled-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/auth-disabled-test-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/auth-disabled-test-kserve-shadow-svc
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-route" not found

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f

SuccessfulCreate

Created pod: llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fmxwss

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.Secret kserve-ci-e2e-test/llmisv08544b88a8d963ffd553cc1f3ed82d16-kserve-self-signed-certs

kserve-ci-e2e-test

deployment-controller

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-sa

kserve-ci-e2e-test

multus

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fmxwss

AddedInterface

Add eth0 [10.134.0.22/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheduler-6fcb489785

SuccessfulCreate

Created pod: llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schekdmw4

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test76d7f532acb694e4a7bcef75d32cd8a1-llmisvc-mode

kserve-ci-e2e-test

deployment-controller

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheduler-6fcb489785 from 0 to 1

kserve-ci-e2e-test

multus

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schekdmw4

AddedInterface

Add eth0 [10.134.0.23/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schekdmw4

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schekdmw4

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schekdmw4

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fmxwss

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fmxwss

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fmxwss

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv08544b88a8d963ffd553cc1f3ed82d16-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-qwen2-5-0-5b-rout-4f8c0978] is Ready

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fmxwss

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schekdmw4

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schekdmw4

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test76d7f532acb694e4a7bcef75d32cd8a1-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-5np5b

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep77sg

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep77sg

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test8ac8e3d2264ccb939eb021b0b835847c-llmisvc-rout
(x11)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-with-7ca60146

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route" not found

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheduler-6f4dcf465

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schepcpc7

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-service

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-6c94dd65d4-jd986

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-6c94dd65d4-jd986

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-6c94dd65d4-jd986

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-6c94dd65d4-jd986

AddedInterface

Add eth0 [10.133.0.55/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test5216bfd716f919dc046bc693ceb22e41-llmisvc-mode

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.Secret kserve-ci-e2e-test/llmisv3e414c2ba058a022dfd694dbcbac5b51-kserve-self-signed-certs

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-with-7ca60146-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-with-7ca60146-kserve-6c94dd65d4 from 0 to 1

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheduler-6f4dcf465 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-6c94dd65d4

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-with-7ca60146-kserve-6c94dd65d4-jd986

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schepcpc7

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schepcpc7

Started

Started container storage-initializer

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schepcpc7

AddedInterface

Add eth0 [10.132.0.45/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schepcpc7

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schepcpc7

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schepcpc7

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schepcpc7

Created

Created container: main
(x12)

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-84ff994f99-sl6v4

Unhealthy

Startup probe failed: Get "https://10.133.0.53:8000/health": dial tcp 10.133.0.53:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-6c94dd65d4-jd986

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-6c94dd65d4-jd986

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-6c94dd65d4-jd986

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv3e414c2ba058a022dfd694dbcbac5b51-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

LLMInferenceServiceReady

LLMInferenceService [auth-disabled-test] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-disabled-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-router-scheduler-7575d494fd-whhqz

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-84ff994f99-sl6v4

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-router-scheduler-7575d494fd-whhqz

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-84ff994f99-sl6v4

Unhealthy

Readiness probe failed: Get "https://10.133.0.53:8000/health": dial tcp 10.133.0.53:8000: connect: connection refused

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-799f46c59b

SuccessfulCreate

Created pod: router-gateway-1-openshift-default-799f46c59b-44gcv

kserve-ci-e2e-test

multus

router-gateway-1-openshift-default-799f46c59b-44gcv

AddedInterface

Add eth0 [10.134.0.24/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-44gcv

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8"

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-799f46c59b

SuccessfulDelete

Deleted pod: router-gateway-1-openshift-default-799f46c59b-44gcv

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-44gcv

Started

Started container istio-proxy

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-44gcv

Killing

Stopping container istio-proxy

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-44gcv

Created

Created container: istio-proxy

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-44gcv

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" in 2.685s (2.685s including waiting). Image size: 181251556 bytes.
(x11)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-e95b1dc1

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-e95b1dc1

Created

Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-e95b1dc1

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-workload-svc

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4wjhf9

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-e95b1dc1

Created

Created v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-e95b1dc1

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-e95b1dc1

Created

Created v1.Secret kserve-ci-e2e-test/llmisv122f03714c5bdf915a2917fdf1262b98-kserve-self-signed-certs

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4wjhf9

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4wjhf9

AddedInterface

Add eth0 [10.134.0.25/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4 from 0 to 1

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4wjhf9

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4wjhf9

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-e95b1dc1

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv122f03714c5bdf915a2917fdf1262b98-kserve-self-signed-certs
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-rxfpz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-rxfpz

Created

Created container: istio-proxy
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-rxfpz

Started

Started container istio-proxy

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-e95b1dc1

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-fb-opt-125m-route-e95b1dc1] is Ready
(x6)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-rxfpz

Unhealthy

Startup probe failed: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused
(x8)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-6c94dd65d4-jd986

Unhealthy

Startup probe failed: Get "https://10.133.0.55:8000/health": dial tcp 10.133.0.55:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-fb-opt-125m-with-7ca60146] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test5216bfd716f919dc046bc693ceb22e41-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-6c94dd65d4-jd986

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schepcpc7

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schepcpc7

Killing

Stopping container main
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-with-ba4d693a

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.Secret kserve-ci-e2e-test/llmisv77ff2528d3e9b4972cd9335229fce9f0-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test05addb65ba05195619f26ef266e8fc04-llmisvc-mode

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-role

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-7dfc79f5dc from 0 to 1

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-7dfc79f5dc

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-7dfc79f5dc-wvz4n

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schen2sbt

Started

Started container storage-initializer

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-7dfc79f5dc-wvz4n

AddedInterface

Add eth0 [10.133.0.56/23] from ovn-kubernetes

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schen2sbt

AddedInterface

Add eth0 [10.132.0.46/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-scheduler-5f4547675b from 0 to 1

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-scheduler-5f4547675b

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schen2sbt

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schen2sbt

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schen2sbt

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-7dfc79f5dc-wvz4n

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-7dfc79f5dc-wvz4n

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-scheduler

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-7dfc79f5dc-wvz4n

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-service

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schen2sbt

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schen2sbt

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schen2sbt

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv77ff2528d3e9b4972cd9335229fce9f0-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-7dfc79f5dc-wvz4n

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-7dfc79f5dc-wvz4n

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-7dfc79f5dc-wvz4n

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine
(x7)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-7dfc79f5dc-wvz4n

Unhealthy

Startup probe failed: Get "https://10.133.0.56:8000/health": dial tcp 10.133.0.56:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-fb-opt-125m-with-ba4d693a] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test05addb65ba05195619f26ef266e8fc04-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schen2sbt

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schen2sbt

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-7dfc79f5dc-wvz4n

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-7dfc79f5dc-wvz4n

Unhealthy

Readiness probe failed: Get "https://10.133.0.56:8000/health": dial tcp 10.133.0.56:8000: connect: connection refused

kserve-ci-e2e-test

ebs.csi.aws.com_aws-ebs-csi-driver-controller-6f9c858f49-qw5h8_27915ec2-eed3-423c-86c6-596d130af027

e2e-pvc-model-storage

Provisioning

External provisioner is provisioning volume for claim "kserve-ci-e2e-test/e2e-pvc-model-storage"

kserve-ci-e2e-test

job-controller

e2e-pvc-model-download

SuccessfulCreate

Created pod: e2e-pvc-model-download-bsd5v

kserve-ci-e2e-test

persistentvolume-controller

e2e-pvc-model-storage

WaitForFirstConsumer

waiting for first consumer to be created before binding
(x2)

kserve-ci-e2e-test

persistentvolume-controller

e2e-pvc-model-storage

ExternalProvisioning

Waiting for a volume to be created either by the external provisioner 'ebs.csi.aws.com' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered.

kserve-ci-e2e-test

ebs.csi.aws.com_aws-ebs-csi-driver-controller-6f9c858f49-qw5h8_27915ec2-eed3-423c-86c6-596d130af027

e2e-pvc-model-storage

ProvisioningSucceeded

Successfully provisioned volume pvc-6394c2ef-5d09-4573-b34c-dae5966e5434

kserve-ci-e2e-test

attachdetach-controller

e2e-pvc-model-download-bsd5v

SuccessfulAttachVolume

AttachVolume.Attach succeeded for volume "pvc-6394c2ef-5d09-4573-b34c-dae5966e5434"

kserve-ci-e2e-test

kubelet

e2e-pvc-model-download-bsd5v

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

multus

e2e-pvc-model-download-bsd5v

AddedInterface

Add eth0 [10.132.0.47/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

e2e-pvc-model-download-bsd5v

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

e2e-pvc-model-download-bsd5v

Created

Created container: storage-initializer

kserve-ci-e2e-test

job-controller

e2e-pvc-model-download

Completed

Job completed

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.Secret kserve-ci-e2e-test/llmisv8bf079eb6eda4debfb4ef5bb7817824c-kserve-self-signed-certs

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9d

SuccessfulCreate

Created pod: llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9df8wz4

kserve-ci-e2e-test

deployment-controller

llmisvc-model-pvc-router-manage-59b9d263-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9d from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-testc9569cf4801efc0ed27b2f25ffaee875-llmisvc-mode

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-epp-sa
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-pvc-router-manage-59b9d263

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-pvc-router-manage-59b9d263-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-pvc-router-manage-59b9d263-kserve-route" not found

kserve-ci-e2e-test

deployment-controller

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheduler-5f555d4d85 from 0 to 1

kserve-ci-e2e-test

kubelet

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schehqcg7

Started

Started container main

kserve-ci-e2e-test

multus

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schehqcg7

AddedInterface

Add eth0 [10.132.0.48/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-epp-service

kserve-ci-e2e-test

kubelet

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schehqcg7

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schehqcg7

Created

Created container: main

kserve-ci-e2e-test

replicaset-controller

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheduler-5f555d4d85

SuccessfulCreate

Created pod: llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schehqcg7

kserve-ci-e2e-test

attachdetach-controller

llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9df8wz4

SuccessfulAttachVolume

AttachVolume.Attach succeeded for volume "pvc-6394c2ef-5d09-4573-b34c-dae5966e5434"

kserve-ci-e2e-test

attachdetach-controller

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schehqcg7

FailedAttachVolume

Multi-Attach error for volume "pvc-6394c2ef-5d09-4573-b34c-dae5966e5434" Volume is already used by pod(s) llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9df8wz4

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9df8wz4

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9df8wz4

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9df8wz4

Created

Created container: main

kserve-ci-e2e-test

multus

llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9df8wz4

AddedInterface

Add eth0 [10.133.0.57/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-shadow-svc
(x2)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv8bf079eb6eda4debfb4ef5bb7817824c-kserve-self-signed-certs
(x10)

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9df8wz4

Unhealthy

Startup probe failed: Get "https://10.133.0.57:8000/health": dial tcp 10.133.0.57:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4wjhf9

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-pvc-router-manage-59b9d263] is Ready

kserve-ci-e2e-test

kubelet

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schehqcg7

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schehqcg7

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9df8wz4

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-testc9569cf4801efc0ed27b2f25ffaee875-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4wjhf9

Unhealthy

Readiness probe failed: Get "https://10.134.0.25:8000/health": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
(x10)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-87882a8e

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-87882a8e-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-87882a8e-kserve-route" not found

kserve-ci-e2e-test

replicaset-controller

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheduler-5dccc4965f

SuccessfulCreate

Created pod: llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche58jw2

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-epp-service

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-66b95f57df

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-87882a8e-kserve-66b95f57dfnrhp7

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-66b95f57dfnrhp7

AddedInterface

Add eth0 [10.133.0.58/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-66b95f57dfnrhp7

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-66b95f57dfnrhp7

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-66b95f57dfnrhp7

Started

Started container storage-initializer

kserve-ci-e2e-test

multus

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche58jw2

AddedInterface

Add eth0 [10.132.0.49/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheduler

kserve-ci-e2e-test

deployment-controller

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheduler-5dccc4965f from 0 to 1

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-87882a8e-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-87882a8e-kserve-66b95f57df from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.Secret kserve-ci-e2e-test/llmisva690bbc929faec8bc98c767f16c003c1-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test21fe6730fe484f3a92b1a16afe1bac8f-llmisvc-mode

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-epp-rb

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche58jw2

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche58jw2

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche58jw2

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche58jw2

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche58jw2

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche58jw2

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9df8wz4

Unhealthy

Readiness probe failed: Get "https://10.133.0.57:8000/health": dial tcp 10.133.0.57:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-66b95f57dfnrhp7

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-66b95f57dfnrhp7

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-66b95f57dfnrhp7

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisva690bbc929faec8bc98c767f16c003c1-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.Secret kserve-ci-e2e-test/llmisvb19f98874e050eec8ca94d49676113f0-kserve-self-signed-certs
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-pvc-router-manage-e8706282

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-pvc-router-manage-e8706282-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-pvc-router-manage-e8706282-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve

kserve-ci-e2e-test

multus

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schegd925

AddedInterface

Add eth0 [10.134.0.26/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-prefill

kserve-ci-e2e-test

deployment-controller

llmisvc-model-pvc-router-manage-e8706282-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c from 0 to 1

kserve-ci-e2e-test

deployment-controller

llmisvc-model-pvc-router-manage-e8706282-kserve-prefill

ScalingReplicaSet

Scaled up replica set llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79497db4cc from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-testf5d060a5eb39a04e074b78907a1556a6-llmisvc-mode

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c

SuccessfulCreate

Created pod: llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-2hv7k

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79497db4cc

SuccessfulCreate

Created pod: llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-795tshv

kserve-ci-e2e-test

replicaset-controller

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheduler-67b4bb9646

SuccessfulCreate

Created pod: llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schegd925

kserve-ci-e2e-test

deployment-controller

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheduler-67b4bb9646 from 0 to 1

kserve-ci-e2e-test

kubelet

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schegd925

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schegd925

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schegd925

Started

Started container main

kserve-ci-e2e-test

attachdetach-controller

llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-795tshv

SuccessfulAttachVolume

AttachVolume.Attach succeeded for volume "pvc-6394c2ef-5d09-4573-b34c-dae5966e5434"

kserve-ci-e2e-test

attachdetach-controller

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-2hv7k

SuccessfulAttachVolume

AttachVolume.Attach succeeded for volume "pvc-6394c2ef-5d09-4573-b34c-dae5966e5434"

kserve-ci-e2e-test

attachdetach-controller

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schegd925

FailedAttachVolume

Multi-Attach error for volume "pvc-6394c2ef-5d09-4573-b34c-dae5966e5434" Volume is already used by pod(s) llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-2hv7k, llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-795tshv

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-795tshv

Started

Started container main

kserve-ci-e2e-test

multus

llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-795tshv

AddedInterface

Add eth0 [10.133.0.60/23] from ovn-kubernetes

kserve-ci-e2e-test

multus

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-2hv7k

AddedInterface

Add eth0 [10.133.0.59/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-2hv7k

Pulling

Pulling image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2"

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-795tshv

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-795tshv

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-2hv7k

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" in 782ms (782ms including waiting). Image size: 44914394 bytes.

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-2hv7k

Started

Started container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-2hv7k

Created

Created container: llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-2hv7k

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-2hv7k

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-2hv7k

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-route
(x10)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-shadow-svc
(x2)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisvb19f98874e050eec8ca94d49676113f0-kserve-self-signed-certs
(x15)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-66b95f57dfnrhp7

Unhealthy

Startup probe failed: Get "https://10.133.0.58:8000/health": dial tcp 10.133.0.58:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-fb-opt-125m-route-87882a8e] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test21fe6730fe484f3a92b1a16afe1bac8f-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche58jw2

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche58jw2

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-66b95f57dfnrhp7

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-66b95f57dfnrhp7

Unhealthy

Readiness probe failed: Get "https://10.133.0.58:8000/health": dial tcp 10.133.0.58:8000: connect: connection refused
(x17)

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-795tshv

Unhealthy

Startup probe failed: Get "https://10.133.0.60:8000/health": dial tcp 10.133.0.60:8000: connect: connection refused
(x17)

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-2hv7k

Unhealthy

Startup probe failed: Get "https://10.133.0.59:8001/health": dial tcp 10.133.0.59:8001: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Service kserve-ci-e2e-test/custom-route-timeout-test-kserve-workload-svc

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-test-kserve-86c9c75755

SuccessfulCreate

Created pod: custom-route-timeout-test-kserve-86c9c75755-7p6kx

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-test-kserve

ScalingReplicaSet

Scaled up replica set custom-route-timeout-test-kserve-86c9c75755 from 0 to 1
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

custom-route-timeout-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy custom-route-timeout-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/custom-route-timeout-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "custom-route-timeout-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Secret kserve-ci-e2e-test/custom-route-timeout-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-test-kserve

kserve-ci-e2e-test

multus

custom-route-timeout-test-kserve-86c9c75755-7p6kx

AddedInterface

Add eth0 [10.133.0.61/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Role kserve-ci-e2e-test/custom-route-timeout-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/custom-route-timeout-test-epp-rb

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-test-kserve-router-scheduler-664966fc9d

SuccessfulCreate

Created pod: custom-route-timeout-test-kserve-router-scheduler-664966fctvzts

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-86c9c75755-7p6kx

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-86c9c75755-7p6kx

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-86c9c75755-7p6kx

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-test-epp-sa

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set custom-route-timeout-test-kserve-router-scheduler-664966fc9d from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Service kserve-ci-e2e-test/custom-route-timeout-test-epp-service

kserve-ci-e2e-test

multus

custom-route-timeout-test-kserve-router-scheduler-664966fctvzts

AddedInterface

Add eth0 [10.132.0.50/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-664966fctvzts

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-664966fctvzts

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-664966fctvzts

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-664966fctvzts

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-664966fctvzts

Created

Created container: main

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-664966fctvzts

Started

Started container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-86c9c75755-7p6kx

Started

Started container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-86c9c75755-7p6kx

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-86c9c75755-7p6kx

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Updated

Updated v1.Secret kserve-ci-e2e-test/custom-route-timeout-test-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/custom-route-timeout-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/custom-route-timeout-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-pvc-router-manage-e8706282] is Ready

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-795tshv

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-testf5d060a5eb39a04e074b78907a1556a6-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schegd925

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-2hv7k

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schegd925

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-2hv7k

Killing

Stopping container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-795tshv

Unhealthy

Readiness probe failed: Get "https://10.133.0.60:8000/health": dial tcp 10.133.0.60:8000: connect: connection refused
(x10)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-pvc-router-manage-2577e794

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-pvc-router-manage-2577e794-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-pvc-router-manage-2577e794-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.LeaderWorkerSet kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.Secret kserve-ci-e2e-test/llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-scc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn

kserve-ci-e2e-test

kubelet

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-schezjgtq

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-epp-rb

kserve-ci-e2e-test

multus

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-schezjgtq

AddedInterface

Add eth0 [10.134.0.27/23] from ovn-kubernetes

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-pvc-router-manage-2577e794-kserve-mn

GroupsProgressing

Created worker statefulset for leader pod llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

kserve-ci-e2e-test

kubelet

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-schezjgtq

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

replicaset-controller

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheduler-7dbcb75dbc

SuccessfulCreate

Created pod: llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-schezjgtq

kserve-ci-e2e-test

statefulset-controller

llmisvc-model-pvc-router-manage-2577e794-kserve-mn

SuccessfulCreate

create Pod llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0 in StatefulSet llmisvc-model-pvc-router-manage-2577e794-kserve-mn successful

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-pvc-router-manage-2577e794-kserve-mn

GroupsProgressing

Replicas are progressing, with 0 groups ready of total 1 groups

kserve-ci-e2e-test

kubelet

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-schezjgtq

Created

Created container: main

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-pvc-router-manage-2577e794-kserve-mn

GroupsProgressing

Created leader statefulset llmisvc-model-pvc-router-manage-2577e794-kserve-mn

kserve-ci-e2e-test

deployment-controller

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheduler-7dbcb75dbc from 0 to 1

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-pvc-router-manage-2577e794-kserve-mn

CreatingRevision

Creating revision with key 686d468674 for a newly created LeaderWorkerSet

kserve-ci-e2e-test

statefulset-controller

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

SuccessfulCreate

create Pod llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1 in StatefulSet llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0 successful

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test37591b20e96e9663d45a730d03070f1e-llmisvc-mode

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-epp-role

kserve-ci-e2e-test

attachdetach-controller

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-schezjgtq

FailedAttachVolume

Multi-Attach error for volume "pvc-6394c2ef-5d09-4573-b34c-dae5966e5434" Volume is already used by pod(s) llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0, llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1

kserve-ci-e2e-test

attachdetach-controller

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

SuccessfulAttachVolume

AttachVolume.Attach succeeded for volume "pvc-6394c2ef-5d09-4573-b34c-dae5966e5434"

kserve-ci-e2e-test

multus

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1

AddedInterface

Add eth0 [10.133.0.63/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

Created

Created container: main

kserve-ci-e2e-test

multus

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

AddedInterface

Add eth0 [10.133.0.62/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-route
(x8)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-shadow-svc
(x2)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs
(x15)

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-86c9c75755-7p6kx

Unhealthy

Startup probe failed: Get "https://10.133.0.61:8000/health": dial tcp 10.133.0.61:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

LLMInferenceServiceReady

LLMInferenceService [custom-route-timeout-test] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-664966fctvzts

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-664966fctvzts

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-86c9c75755-7p6kx

Killing

Stopping container main
(x2)

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-86c9c75755-7p6kx

Unhealthy

Readiness probe failed: Get "https://10.133.0.61:8000/health": dial tcp 10.133.0.61:8000: connect: connection refused
(x4)

kserve-ci-e2e-test

gateway_labeler_controller

router-gateway-1

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway router-gateway-1
(x4)

kserve-ci-e2e-test

deployment-controller

router-gateway-1-openshift-default

ScalingReplicaSet

Scaled up replica set router-gateway-1-openshift-default-799f46c59b from 0 to 1

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-799f46c59b

SuccessfulCreate

Created pod: router-gateway-1-openshift-default-799f46c59b-z6cd2

kserve-ci-e2e-test

multus

router-gateway-1-openshift-default-799f46c59b-z6cd2

AddedInterface

Add eth0 [10.132.0.51/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-z6cd2

Created

Created container: istio-proxy

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-z6cd2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-z6cd2

Started

Started container istio-proxy

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-799f46c59b

SuccessfulDelete

Deleted pod: router-gateway-1-openshift-default-799f46c59b-z6cd2
(x4)

kserve-ci-e2e-test

deployment-controller

router-gateway-1-openshift-default

ScalingReplicaSet

Scaled down replica set router-gateway-1-openshift-default-799f46c59b from 1 to 0

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-z6cd2

Killing

Stopping container istio-proxy

kserve-ci-e2e-test

replicaset-controller

router-with-refs-test-kserve-697958d7

SuccessfulCreate

Created pod: router-with-refs-test-kserve-697958d7-hhdqd

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Service kserve-ci-e2e-test/router-with-refs-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Deployment kserve-ci-e2e-test/router-with-refs-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Secret kserve-ci-e2e-test/router-with-refs-test-kserve-self-signed-certs

kserve-ci-e2e-test

deployment-controller

router-with-refs-test-kserve

ScalingReplicaSet

Scaled up replica set router-with-refs-test-kserve-697958d7 from 0 to 1

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-697958d7-hhdqd

Created

Created container: storage-initializer

kserve-ci-e2e-test

deployment-controller

router-with-refs-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set router-with-refs-test-kserve-router-scheduler-85b8956674 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Service kserve-ci-e2e-test/router-with-refs-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Deployment kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler

kserve-ci-e2e-test

multus

router-with-refs-test-kserve-router-scheduler-85b8956674-9vvqt

AddedInterface

Add eth0 [10.132.0.52/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-85b8956674-9vvqt

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-85b8956674-9vvqt

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-85b8956674-9vvqt

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/router-with-refs-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Role kserve-ci-e2e-test/router-with-refs-test-epp-role

kserve-ci-e2e-test

multus

router-with-refs-test-kserve-697958d7-hhdqd

AddedInterface

Add eth0 [10.133.0.64/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

router-with-refs-test-kserve-router-scheduler-85b8956674

SuccessfulCreate

Created pod: router-with-refs-test-kserve-router-scheduler-85b8956674-9vvqt
(x16)

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

Unhealthy

Startup probe failed: Get "https://10.133.0.62:8000/health": dial tcp 10.133.0.62:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-697958d7-hhdqd

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-697958d7-hhdqd

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-85b8956674-9vvqt

Started

Started container main

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-85b8956674-9vvqt

Created

Created container: main

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-85b8956674-9vvqt

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine
(x5)

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/router-with-refs-test-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-697958d7-hhdqd

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-697958d7-hhdqd

Started

Started container main

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-697958d7-hhdqd

Created

Created container: main
(x2)

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-pvc-router-manage-2577e794-kserve-mn

AllGroupsReady

All replicas are ready, with 1 groups ready of total 1 groups

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Updated

Updated v1.Secret kserve-ci-e2e-test/router-with-refs-test-kserve-self-signed-certs
(x11)

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-697958d7-hhdqd

Unhealthy

Startup probe failed: Get "https://10.133.0.64:8000/health": dial tcp 10.133.0.64:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

LLMInferenceServiceReady

LLMInferenceService [router-with-refs-test] is Ready

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-697958d7-hhdqd

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-85b8956674-9vvqt

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-85b8956674-9vvqt

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-697958d7-hhdqd

Unhealthy

Readiness probe failed: Get "https://10.133.0.64:8000/health": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
(x10)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-50bc673d

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-50bc673d-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-50bc673d-kserve-route" not found

kserve-ci-e2e-test

deployment-controller

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheduler-5d8499d949 from 0 to 1

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-596f5b7f78 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-59596f6d4f

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-50bc673d-kserve-59596f6d4fsgc99

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-59596f6d4fsgc99

AddedInterface

Add eth0 [10.133.0.65/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5989gpd

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5989gpd

Created

Created container: storage-initializer

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5989gpd

AddedInterface

Add eth0 [10.133.0.66/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-596f5b7f78

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5989gpd

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-epp-sa

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-59596f6d4fsgc99

Pulled

Container image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-59596f6d4fsgc99

Started

Started container llm-d-routing-sidecar

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test00d7278d8a22c4e39146a6b0eb840f45-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-59596f6d4fsgc99

Created

Created container: llm-d-routing-sidecar

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.Secret kserve-ci-e2e-test/llmisv44d181485fad85e662eb092f3749502f-kserve-self-signed-certs

kserve-ci-e2e-test

replicaset-controller

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheduler-5d8499d949

SuccessfulCreate

Created pod: llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schepcckr

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-50bc673d-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-50bc673d-kserve-59596f6d4f from 0 to 1

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5989gpd

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-59596f6d4fsgc99

Created

Created container: storage-initializer

kserve-ci-e2e-test

multus

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schepcckr

AddedInterface

Add eth0 [10.132.0.53/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-59596f6d4fsgc99

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-59596f6d4fsgc99

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schepcckr

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schepcckr

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schepcckr

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schepcckr

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schepcckr

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schepcckr

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-pvc-router-manage-2577e794] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv44d181485fad85e662eb092f3749502f-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-59596f6d4fsgc99

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-route

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-59596f6d4fsgc99

Created

Created container: main
(x10)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-59596f6d4fsgc99

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5989gpd

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5989gpd

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5989gpd

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-schezjgtq

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test37591b20e96e9663d45a730d03070f1e-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-schezjgtq

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

Unhealthy

Readiness probe failed: Get "https://10.133.0.62:8000/health": dial tcp 10.133.0.62:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

conv-test-criticality

UpdateFailed

Failed to update status for LLMInferenceService "conv-test-criticality": llminferenceservices.serving.kserve.io "conv-test-criticality" not found

kserve-ci-e2e-test

LLMInferenceServiceController

conv-test-lora-crit

UpdateFailed

Failed to update status for LLMInferenceService "conv-test-lora-crit": llminferenceservices.serving.kserve.io "conv-test-lora-crit" not found

kserve-ci-e2e-test

LLMInferenceServiceController

conv-test-round-trip

UpdateFailed

Failed to update status for LLMInferenceService "conv-test-round-trip": llminferenceservices.serving.kserve.io "conv-test-round-trip" not found

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler

kserve-ci-e2e-test

replicaset-controller

stop-feature-test-kserve-7c7845f79f

SuccessfulCreate

Created pod: stop-feature-test-kserve-7c7845f79f-knxs6

kserve-ci-e2e-test

deployment-controller

stop-feature-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set stop-feature-test-kserve-router-scheduler-7cb57d7b84 from 0 to 1

kserve-ci-e2e-test

deployment-controller

stop-feature-test-kserve

ScalingReplicaSet

Scaled up replica set stop-feature-test-kserve-7c7845f79f from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs

kserve-ci-e2e-test

replicaset-controller

stop-feature-test-kserve-router-scheduler-7cb57d7b84

SuccessfulCreate

Created pod: stop-feature-test-kserve-router-scheduler-7cb57d7b84-rhxsb

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Service kserve-ci-e2e-test/stop-feature-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/stop-feature-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Role kserve-ci-e2e-test/stop-feature-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-stop-feature-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/stop-feature-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Service kserve-ci-e2e-test/stop-feature-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve

kserve-ci-e2e-test

multus

stop-feature-test-kserve-router-scheduler-7cb57d7b84-rhxsb

AddedInterface

Add eth0 [10.132.0.54/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7c7845f79f-knxs6

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

multus

stop-feature-test-kserve-7c7845f79f-knxs6

AddedInterface

Add eth0 [10.133.0.67/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-7cb57d7b84-rhxsb

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-7cb57d7b84-rhxsb

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-7cb57d7b84-rhxsb

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7c7845f79f-knxs6

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7c7845f79f-knxs6

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-7cb57d7b84-rhxsb

Created

Created container: main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-7cb57d7b84-rhxsb

Started

Started container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-7cb57d7b84-rhxsb

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7c7845f79f-knxs6

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7c7845f79f-knxs6

Created

Created container: main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7c7845f79f-knxs6

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Updated

Updated v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/stop-feature-test-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/stop-feature-test-kserve-shadow-svc
(x18)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5989gpd

Unhealthy

Startup probe failed: Get "https://10.133.0.66:8000/health": dial tcp 10.133.0.66:8000: connect: connection refused
(x18)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-59596f6d4fsgc99

Unhealthy

Startup probe failed: Get "https://10.133.0.65:8001/health": dial tcp 10.133.0.65:8001: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-fb-opt-125m-route-50bc673d] is Ready

kserve-ci-e2e-test

kubelet

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schepcckr

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schepcckr

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-59596f6d4fsgc99

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-59596f6d4fsgc99

Killing

Stopping container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5989gpd

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test00d7278d8a22c4e39146a6b0eb840f45-llmisvc-mode
(x16)

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7c7845f79f-knxs6

Unhealthy

Startup probe failed: Get "https://10.133.0.67:8000/health": dial tcp 10.133.0.67:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Service kserve-ci-e2e-test/stop-feature-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7c7845f79f-knxs6

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5989gpd

Unhealthy

Readiness probe failed: Get "https://10.133.0.66:8000/health": dial tcp 10.133.0.66:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.InferencePool kserve-ci-e2e-test/stop-feature-test-inference-pool

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-7cb57d7b84-rhxsb

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-7cb57d7b84-rhxsb

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.ServiceAccount kserve-ci-e2e-test/stop-feature-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Service kserve-ci-e2e-test/stop-feature-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

LLMInferenceServiceNotReady

LLMInferenceService [stop-feature-test] is no longer Ready because of: GatewaysReady, HTTPRoutesReady, InferencePoolReady, MainWorkloadReady, PrefillWorkerWorkloadReady, PrefillWorkloadReady, RouterReady, SchedulerWorkloadReady, WorkerWorkloadReady, WorkloadsReady
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

custom-route-timeout-pd-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy custom-route-timeout-pd-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "custom-route-timeout-pd-test-kserve-route" not found
(x23)

kserve-ci-e2e-test

OpenDataHubModelController

stop-feature-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy stop-feature-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/stop-feature-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "stop-feature-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Role kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-role

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-rb

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Secret kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-74789fhwf6

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Service kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-pd-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-pd-test-epp-auth-rb

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-pd-test-kserve

ScalingReplicaSet

Scaled up replica set custom-route-timeout-pd-test-kserve-67588db874 from 0 to 1

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-94b6f6947-rx456

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-94b6f6947-rx456

Created

Created container: storage-initializer

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-pd-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set custom-route-timeout-pd-test-kserve-router-scheduler-7478954df5 from 0 to 1

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-94b6f6947-rx456

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

multus

custom-route-timeout-pd-test-kserve-prefill-94b6f6947-rx456

AddedInterface

Add eth0 [10.133.0.69/23] from ovn-kubernetes

kserve-ci-e2e-test

multus

custom-route-timeout-pd-test-kserve-67588db874-f6b8t

AddedInterface

Add eth0 [10.133.0.68/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-pd-test-kserve-67588db874

SuccessfulCreate

Created pod: custom-route-timeout-pd-test-kserve-67588db874-f6b8t

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-67588db874-f6b8t

Pulled

Container image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-67588db874-f6b8t

Started

Started container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-67588db874-f6b8t

Created

Created container: llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-74789fhwf6

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-74789fhwf6

Created

Created container: storage-initializer

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-pd-test-kserve-prefill-94b6f6947

SuccessfulCreate

Created pod: custom-route-timeout-pd-test-kserve-prefill-94b6f6947-rx456

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-pd-test-kserve-prefill

ScalingReplicaSet

Scaled up replica set custom-route-timeout-pd-test-kserve-prefill-94b6f6947 from 0 to 1

kserve-ci-e2e-test

multus

custom-route-timeout-pd-test-kserve-router-scheduler-74789fhwf6

AddedInterface

Add eth0 [10.132.0.55/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-pd-test-kserve-router-scheduler-7478954df5

SuccessfulCreate

Created pod: custom-route-timeout-pd-test-kserve-router-scheduler-74789fhwf6

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-67588db874-f6b8t

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-67588db874-f6b8t

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-67588db874-f6b8t

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-74789fhwf6

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-74789fhwf6

Created

Created container: main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-74789fhwf6

Started

Started container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-94b6f6947-rx456

Started

Started container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-94b6f6947-rx456

Created

Created container: main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-94b6f6947-rx456

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-67588db874-f6b8t

Created

Created container: main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-67588db874-f6b8t

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-67588db874-f6b8t

Started

Started container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7c7845f79f-tfvwg

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

replicaset-controller

stop-feature-test-kserve-7c7845f79f

SuccessfulCreate

Created pod: stop-feature-test-kserve-7c7845f79f-tfvwg

kserve-ci-e2e-test

multus

stop-feature-test-kserve-7c7845f79f-tfvwg

AddedInterface

Add eth0 [10.133.0.70/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7c7845f79f-tfvwg

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7c7845f79f-tfvwg

Created

Created container: storage-initializer

kserve-ci-e2e-test

replicaset-controller

stop-feature-test-kserve-router-scheduler-7cb57d7b84

SuccessfulCreate

Created pod: stop-feature-test-kserve-router-scheduler-7cb57d7b84-rn8hp

kserve-ci-e2e-test

deployment-controller

stop-feature-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set stop-feature-test-kserve-router-scheduler-7cb57d7b84 from 0 to 1

kserve-ci-e2e-test

deployment-controller

stop-feature-test-kserve

ScalingReplicaSet

Scaled up replica set stop-feature-test-kserve-7c7845f79f from 0 to 1

kserve-ci-e2e-test

multus

stop-feature-test-kserve-router-scheduler-7cb57d7b84-rn8hp

AddedInterface

Add eth0 [10.132.0.56/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-7cb57d7b84-rn8hp

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-7cb57d7b84-rn8hp

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-7cb57d7b84-rn8hp

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-7cb57d7b84-rn8hp

Started

Started container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-7cb57d7b84-rn8hp

Created

Created container: main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-7cb57d7b84-rn8hp

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7c7845f79f-tfvwg

Started

Started container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7c7845f79f-tfvwg

Created

Created container: main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7c7845f79f-tfvwg

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Updated

Updated v1.Secret kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-self-signed-certs
(x10)

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-route
(x20)

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-67588db874-f6b8t

Unhealthy

Startup probe failed: Get "https://10.133.0.68:8001/health": dial tcp 10.133.0.68:8001: connect: connection refused
(x20)

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-94b6f6947-rx456

Unhealthy

Startup probe failed: Get "https://10.133.0.69:8000/health": dial tcp 10.133.0.69:8000: connect: connection refused
(x20)

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7c7845f79f-tfvwg

Unhealthy

Startup probe failed: Get "https://10.133.0.70:8000/health": dial tcp 10.133.0.70:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

LLMInferenceServiceReady

LLMInferenceService [custom-route-timeout-pd-test] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-pd-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-74789fhwf6

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-67588db874-f6b8t

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-74789fhwf6

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-67588db874-f6b8t

Killing

Stopping container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-94b6f6947-rx456

Killing

Stopping container main
(x2)

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

LLMInferenceServiceReady

LLMInferenceService [stop-feature-test] is Ready

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-7cb57d7b84-rn8hp

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7c7845f79f-tfvwg

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-7cb57d7b84-rn8hp

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

deployment-controller

router-gateway-2-openshift-default

ScalingReplicaSet

Scaled up replica set router-gateway-2-openshift-default-54c789bdc6 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

router-gateway-2-openshift-default-684dc9b4b9

SuccessfulCreate

Created pod: router-gateway-2-openshift-default-684dc9b4b9-h2x2d

kserve-ci-e2e-test

gateway_labeler_controller

router-gateway-2

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway router-gateway-2

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-54c789bdc6-tppn4

Started

Started container istio-proxy

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-54c789bdc6-tppn4

Created

Created container: istio-proxy

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-54c789bdc6-tppn4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine

kserve-ci-e2e-test

multus

router-gateway-2-openshift-default-684dc9b4b9-h2x2d

AddedInterface

Add eth0 [10.133.0.71/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

router-gateway-2-openshift-default

ScalingReplicaSet

Scaled up replica set router-gateway-2-openshift-default-684dc9b4b9 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

router-gateway-2-openshift-default-54c789bdc6

SuccessfulCreate

Created pod: router-gateway-2-openshift-default-54c789bdc6-tppn4

kserve-ci-e2e-test

multus

router-gateway-2-openshift-default-54c789bdc6-tppn4

AddedInterface

Add eth0 [10.134.0.28/23] from ovn-kubernetes

kserve-ci-e2e-test

service-controller

router-gateway-2-openshift-default

EnsuringLoadBalancer

Ensuring load balancer

kserve-ci-e2e-test

replicaset-controller

router-gateway-2-openshift-default-54c789bdc6

SuccessfulDelete

Deleted pod: router-gateway-2-openshift-default-54c789bdc6-tppn4

kserve-ci-e2e-test

deployment-controller

router-gateway-2-openshift-default

ScalingReplicaSet

Scaled down replica set router-gateway-2-openshift-default-54c789bdc6 from 1 to 0

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-54c789bdc6-tppn4

Killing

Stopping container istio-proxy

kserve-ci-e2e-test

service-controller

router-gateway-2-openshift-default

EnsuredLoadBalancer

Ensured load balancer
(x11)

kserve-ci-e2e-test

OpenDataHubModelController

lora-single-adapter-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy lora-single-adapter-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/lora-single-adapter-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "lora-single-adapter-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

lora-single-adapter-test

Created

Created v1.HTTPRoute kserve-ci-e2e-test/lora-single-adapter-test-kserve-route

kserve-ci-e2e-test

deployment-controller

lora-single-adapter-test-kserve

ScalingReplicaSet

Scaled up replica set lora-single-adapter-test-kserve-75fdddbc5d from 0 to 1

kserve-ci-e2e-test

replicaset-controller

lora-single-adapter-test-kserve-75fdddbc5d

SuccessfulCreate

Created pod: lora-single-adapter-test-kserve-75fdddbc5d-r8z7x

kserve-ci-e2e-test

LLMInferenceServiceController

lora-single-adapter-test

Created

Created v1.DestinationRule kserve-ci-e2e-test/lora-single-adapter-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

lora-single-adapter-test

Created

Created v1.Service kserve-ci-e2e-test/lora-single-adapter-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

lora-single-adapter-test

Created

Created v1.Deployment kserve-ci-e2e-test/lora-single-adapter-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

lora-single-adapter-test

Created

Created v1.Secret kserve-ci-e2e-test/lora-single-adapter-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-75fdddbc5d-r8z7x

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-75fdddbc5d-r8z7x

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-75fdddbc5d-r8z7x

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

multus

lora-single-adapter-test-kserve-75fdddbc5d-r8z7x

AddedInterface

Add eth0 [10.133.0.72/23] from ovn-kubernetes
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-684dc9b4b9-h2x2d

Started

Started container istio-proxy
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-684dc9b4b9-h2x2d

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-684dc9b4b9-h2x2d

Created

Created container: istio-proxy

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-75fdddbc5d-r8z7x

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-75fdddbc5d-r8z7x

Created

Created container: main

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-75fdddbc5d-r8z7x

Started

Started container main

kserve-ci-e2e-test

replicaset-controller

router-with-refs-pd-test-kserve-57d4965598

SuccessfulCreate

Created pod: router-with-refs-pd-test-kserve-57d4965598-p2stc

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Secret kserve-ci-e2e-test/router-with-refs-pd-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-pd-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/router-with-refs-pd-test-kserve-rb

kserve-ci-e2e-test

multus

router-with-refs-pd-test-kserve-57d4965598-p2stc

AddedInterface

Add eth0 [10.133.0.73/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Deployment kserve-ci-e2e-test/router-with-refs-pd-test-kserve

kserve-ci-e2e-test

deployment-controller

router-with-refs-pd-test-kserve-prefill

ScalingReplicaSet

Scaled up replica set router-with-refs-pd-test-kserve-prefill-5c9fc7cc79 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Role kserve-ci-e2e-test/router-with-refs-pd-test-kserve-role

kserve-ci-e2e-test

deployment-controller

router-with-refs-pd-test-kserve

ScalingReplicaSet

Scaled up replica set router-with-refs-pd-test-kserve-57d4965598 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

router-with-refs-pd-test-kserve-prefill-5c9fc7cc79

SuccessfulCreate

Created pod: router-with-refs-pd-test-kserve-prefill-5c9fc7cc79-2ggqn

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Deployment kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Service kserve-ci-e2e-test/router-with-refs-pd-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-pd-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-pd-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-68dbc8995xw9h6

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-57d4965598-p2stc

Started

Started container storage-initializer
(x6)

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-684dc9b4b9-h2x2d

Unhealthy

Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-57d4965598-p2stc

Pulled

Container image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-57d4965598-p2stc

Created

Created container: llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-57d4965598-p2stc

Started

Started container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-57d4965598-p2stc

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-57d4965598-p2stc

Created

Created container: storage-initializer

kserve-ci-e2e-test

multus

router-with-refs-pd-test-kserve-prefill-5c9fc7cc79-2ggqn

AddedInterface

Add eth0 [10.133.0.74/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5c9fc7cc79-2ggqn

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5c9fc7cc79-2ggqn

Created

Created container: storage-initializer

kserve-ci-e2e-test

deployment-controller

router-with-refs-pd-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set router-with-refs-pd-test-kserve-router-scheduler-68dbc89955 from 0 to 1

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5c9fc7cc79-2ggqn

Started

Started container storage-initializer

kserve-ci-e2e-test

replicaset-controller

router-with-refs-pd-test-kserve-router-scheduler-68dbc89955

SuccessfulCreate

Created pod: router-with-refs-pd-test-kserve-router-scheduler-68dbc8995xw9h6

kserve-ci-e2e-test

multus

router-with-refs-pd-test-kserve-router-scheduler-68dbc8995xw9h6

AddedInterface

Add eth0 [10.132.0.57/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-68dbc8995xw9h6

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-68dbc8995xw9h6

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine
(x9)

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/router-with-refs-pd-test-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-68dbc8995xw9h6

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-68dbc8995xw9h6

Created

Created container: main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-68dbc8995xw9h6

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

lora-single-adapter-test

Updated

Updated v1.Secret kserve-ci-e2e-test/lora-single-adapter-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5c9fc7cc79-2ggqn

Started

Started container main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5c9fc7cc79-2ggqn

Created

Created container: main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5c9fc7cc79-2ggqn

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-57d4965598-p2stc

Created

Created container: main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-57d4965598-p2stc

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-57d4965598-p2stc

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Updated

Updated v1.Secret kserve-ci-e2e-test/router-with-refs-pd-test-kserve-self-signed-certs
(x8)

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-75fdddbc5d-r8z7x

Unhealthy

Startup probe failed: Get "https://10.133.0.72:8000/health": dial tcp 10.133.0.72:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

lora-single-adapter-test

LLMInferenceServiceReady

LLMInferenceService [lora-single-adapter-test] is Ready

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-75fdddbc5d-r8z7x

Killing

Stopping container main
(x18)

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5c9fc7cc79-2ggqn

Unhealthy

Startup probe failed: Get "https://10.133.0.74:8000/health": dial tcp 10.133.0.74:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-75fdddbc5d-r8z7x

Unhealthy

Readiness probe failed: Get "https://10.133.0.72:8000/health": dial tcp 10.133.0.72:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-75fdddbc5d-r8z7x

Unhealthy

Readiness probe failed: Get "https://10.133.0.72:8000/health": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
(x19)

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-57d4965598-p2stc

Unhealthy

Startup probe failed: Get "https://10.133.0.73:8001/health": dial tcp 10.133.0.73:8001: connect: connection refused
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

lora-multiple-adapters-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy lora-multiple-adapters-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "lora-multiple-adapters-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

lora-multiple-adapters-test

Created

Created v1.Service kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

lora-multiple-adapters-test

Created

Created v1.DestinationRule kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

lora-multiple-adapters-test

Created

Created v1.Deployment kserve-ci-e2e-test/lora-multiple-adapters-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

lora-multiple-adapters-test

Created

Created v1.HTTPRoute kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

lora-multiple-adapters-test

Created

Created v1.Secret kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-self-signed-certs

kserve-ci-e2e-test

deployment-controller

lora-multiple-adapters-test-kserve

ScalingReplicaSet

Scaled up replica set lora-multiple-adapters-test-kserve-7c8db66c87 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

lora-multiple-adapters-test-kserve-7c8db66c87

SuccessfulCreate

Created pod: lora-multiple-adapters-test-kserve-7c8db66c87-pzn59

kserve-ci-e2e-test

multus

lora-multiple-adapters-test-kserve-7c8db66c87-pzn59

AddedInterface

Add eth0 [10.133.0.75/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-7c8db66c87-pzn59

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-7c8db66c87-pzn59

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-7c8db66c87-pzn59

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

LLMInferenceServiceReady

LLMInferenceService [router-with-refs-pd-test] is Ready

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-7c8db66c87-pzn59

Started

Started container main

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-7c8db66c87-pzn59

Created

Created container: main

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-7c8db66c87-pzn59

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

lora-multiple-adapters-test

Updated

Updated v1.Secret kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-68dbc8995xw9h6

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-68dbc8995xw9h6

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5c9fc7cc79-2ggqn

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-pd-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-57d4965598-p2stc

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-57d4965598-p2stc

Killing

Stopping container llm-d-routing-sidecar
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-3c960099

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-3c960099-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-3c960099-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-3c960099

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-workload-svc

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-3c960099-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-3c960099-kserve-6d69fc44d8 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-3c960099

Created

Created v1.Secret kserve-ci-e2e-test/llmisvd10b93c8eba12f87c3a350a5cae2ee0c-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-3c960099

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-3c960099-kserve-6d69fc44d8

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-3c960099-kserve-6d69fc44d8842wn

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-3c960099

Created

Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-workload-svc

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-3c960099-kserve-6d69fc44d8842wn

AddedInterface

Add eth0 [10.133.0.76/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-6d69fc44d8842wn

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-3c960099

Created

Created v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-route

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-6d69fc44d8842wn

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-6d69fc44d8842wn

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-6d69fc44d8842wn

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-6d69fc44d8842wn

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-6d69fc44d8842wn

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-3c960099

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisvd10b93c8eba12f87c3a350a5cae2ee0c-kserve-self-signed-certs
(x7)

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-7c8db66c87-pzn59

Unhealthy

Startup probe failed: Get "https://10.133.0.75:8000/health": dial tcp 10.133.0.75:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

lora-multiple-adapters-test

LLMInferenceServiceReady

LLMInferenceService [lora-multiple-adapters-test] is Ready

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-7c8db66c87-pzn59

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-7c8db66c87-pzn59

Unhealthy

Readiness probe failed: Get "https://10.133.0.75:8000/health": dial tcp 10.133.0.75:8000: connect: connection refused
(x11)

kserve-ci-e2e-test

OpenDataHubModelController

tls-verification-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy tls-verification-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/tls-verification-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "tls-verification-test-kserve-route" not found

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-5847b7959f-z68fl

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

replicaset-controller

tls-verification-test-kserve-5847b7959f

SuccessfulCreate

Created pod: tls-verification-test-kserve-5847b7959f-z68fl

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.Service kserve-ci-e2e-test/tls-verification-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.Deployment kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.Deployment kserve-ci-e2e-test/tls-verification-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.Secret kserve-ci-e2e-test/tls-verification-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-tls-verification-test-epp-auth-rb

kserve-ci-e2e-test

deployment-controller

tls-verification-test-kserve

ScalingReplicaSet

Scaled up replica set tls-verification-test-kserve-5847b7959f from 0 to 1

kserve-ci-e2e-test

deployment-controller

tls-verification-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set tls-verification-test-kserve-router-scheduler-5d7755bf56 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

tls-verification-test-kserve-router-scheduler-5d7755bf56

SuccessfulCreate

Created pod: tls-verification-test-kserve-router-scheduler-5d7755bf56-kb4bv

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.Service kserve-ci-e2e-test/tls-verification-test-epp-service

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-5847b7959f-z68fl

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-5847b7959f-z68fl

Started

Started container storage-initializer

kserve-ci-e2e-test

multus

tls-verification-test-kserve-5847b7959f-z68fl

AddedInterface

Add eth0 [10.133.0.77/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.Role kserve-ci-e2e-test/tls-verification-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/tls-verification-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/tls-verification-test-epp-sa

kserve-ci-e2e-test

multus

tls-verification-test-kserve-router-scheduler-5d7755bf56-kb4bv

AddedInterface

Add eth0 [10.132.0.58/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-5d7755bf56-kb4bv

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-5d7755bf56-kb4bv

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-5d7755bf56-kb4bv

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-5d7755bf56-kb4bv

Created

Created container: main

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-5d7755bf56-kb4bv

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-5d7755bf56-kb4bv

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/tls-verification-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Updated

Updated v1.Secret kserve-ci-e2e-test/tls-verification-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-5847b7959f-z68fl

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-5847b7959f-z68fl

Started

Started container main
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/tls-verification-test-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-5847b7959f-z68fl

Created

Created container: main
(x12)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-6d69fc44d8842wn

Unhealthy

Startup probe failed: Get "https://10.133.0.76:8000/health": dial tcp 10.133.0.76:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-3c960099

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-fb-opt-125m-route-3c960099] is Ready

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-6d69fc44d8842wn

Killing

Stopping container main
(x10)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-dc21cb14

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.Secret kserve-ci-e2e-test/llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-scc

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

GroupsProgressing

Created worker statefulset for leader pod llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schebbsmf

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.LeaderWorkerSet kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-epp-rb

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Created

Created container: storage-initializer

kserve-ci-e2e-test

multus

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schebbsmf

AddedInterface

Add eth0 [10.132.0.60/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-scheduler-79ccfb59

SuccessfulCreate

Created pod: llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schebbsmf

kserve-ci-e2e-test

statefulset-controller

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

SuccessfulCreate

create Pod llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1 in StatefulSet llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0 successful

kserve-ci-e2e-test

statefulset-controller

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

SuccessfulCreate

create Pod llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0 in StatefulSet llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn successful

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test386bd5808c5c450f11fd8632e1f821fb-llmisvc-mode

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

AddedInterface

Add eth0 [10.132.0.59/23] from ovn-kubernetes
(x2)

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

GroupsProgressing

Replicas are progressing, with 0 groups ready of total 1 groups

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

AddedInterface

Add eth0 [10.133.0.78/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-scheduler-79ccfb59 from 0 to 1

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

GroupsProgressing

Created leader statefulset llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Started

Started container storage-initializer

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

CreatingRevision

Creating revision with key d6546f6 for a newly created LeaderWorkerSet

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schebbsmf

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schebbsmf

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schebbsmf

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schebbsmf

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schebbsmf

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Pulling

Pulling image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0"

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs
(x8)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Pulled

Successfully pulled image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" in 26.858s (26.858s including waiting). Image size: 3531177328 bytes.

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Created

Created container: main
(x12)

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-5847b7959f-z68fl

Unhealthy

Startup probe failed: Get "https://10.133.0.77:8000/health": dial tcp 10.133.0.77:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

LLMInferenceServiceReady

LLMInferenceService [tls-verification-test] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-tls-verification-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-5d7755bf56-kb4bv

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-5d7755bf56-kb4bv

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-5847b7959f-z68fl

Killing

Stopping container main
(x10)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Unhealthy

Startup probe failed: Get "https://10.133.0.78:8000/health": dial tcp 10.133.0.78:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-5847b7959f-z68fl

Unhealthy

Readiness probe failed: Get "https://10.133.0.77:8000/health": dial tcp 10.133.0.77:8000: connect: connection refused
(x2)

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

AllGroupsReady

All replicas are ready, with 1 groups ready of total 1 groups

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.Deployment kserve-ci-e2e-test/prestop-hook-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.Service kserve-ci-e2e-test/prestop-hook-test-kserve-workload-svc

kserve-ci-e2e-test

replicaset-controller

prestop-hook-test-kserve-5b7d7d79ff

SuccessfulCreate

Created pod: prestop-hook-test-kserve-5b7d7d79ff-52bt2

kserve-ci-e2e-test

deployment-controller

prestop-hook-test-kserve

ScalingReplicaSet

Scaled up replica set prestop-hook-test-kserve-5b7d7d79ff from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.Secret kserve-ci-e2e-test/prestop-hook-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-5b7d7d79ff-52bt2

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-5b7d7d79ff-52bt2

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-5b7d7d79ff-52bt2

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-767d7586b8-gptkj

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-767d7586b8-gptkj

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-767d7586b8-gptkj

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

multus

prestop-hook-test-kserve-router-scheduler-767d7586b8-gptkj

AddedInterface

Add eth0 [10.133.0.79/23] from ovn-kubernetes

kserve-ci-e2e-test

multus

prestop-hook-test-kserve-5b7d7d79ff-52bt2

AddedInterface

Add eth0 [10.132.0.61/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

prestop-hook-test-kserve-router-scheduler-767d7586b8

SuccessfulCreate

Created pod: prestop-hook-test-kserve-router-scheduler-767d7586b8-gptkj

kserve-ci-e2e-test

deployment-controller

prestop-hook-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set prestop-hook-test-kserve-router-scheduler-767d7586b8 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.Service kserve-ci-e2e-test/prestop-hook-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/prestop-hook-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-prestop-hook-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.Role kserve-ci-e2e-test/prestop-hook-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/prestop-hook-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.Deployment kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-767d7586b8-gptkj

Started

Started container main

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-767d7586b8-gptkj

Created

Created container: main

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-767d7586b8-gptkj

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-fb-opt-125m-route-dc21cb14] is Ready

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-5b7d7d79ff-52bt2

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-5b7d7d79ff-52bt2

Created

Created container: main

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-5b7d7d79ff-52bt2

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/prestop-hook-test-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/prestop-hook-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test386bd5808c5c450f11fd8632e1f821fb-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schebbsmf

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schebbsmf

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-inline-config-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-inline-config-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-inline-config-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-inline-config-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-inline-config-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-inline-config-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-inline-config-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-inline-config-test-epp-sa
(x13)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-inline-config-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-inline-config-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-inline-config-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-inline-config-test-kserve-route" not found

kserve-ci-e2e-test

deployment-controller

scheduler-inline-config-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-inline-config-test-kserve-7c6c58cc96 from 0 to 1

kserve-ci-e2e-test

deployment-controller

scheduler-inline-config-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-inline-config-test-kserve-router-scheduler-d9ddf6c4d from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler

kserve-ci-e2e-test

replicaset-controller

scheduler-inline-config-test-kserve-router-scheduler-d9ddf6c4d

SuccessfulCreate

Created pod: scheduler-inline-config-test-kserve-router-scheduler-d9ddfsqtmv

kserve-ci-e2e-test

replicaset-controller

scheduler-inline-config-test-kserve-7c6c58cc96

SuccessfulCreate

Created pod: scheduler-inline-config-test-kserve-7c6c58cc96-rclxb

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-d9ddfsqtmv

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-d9ddfsqtmv

Started

Started container main

kserve-ci-e2e-test

multus

scheduler-inline-config-test-kserve-7c6c58cc96-rclxb

AddedInterface

Add eth0 [10.134.0.29/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-7c6c58cc96-rclxb

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-7c6c58cc96-rclxb

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-7c6c58cc96-rclxb

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-d9ddfsqtmv

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

multus

scheduler-inline-config-test-kserve-router-scheduler-d9ddfsqtmv

AddedInterface

Add eth0 [10.132.0.62/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-inline-config-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-inline-config-test-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-inline-config-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

LLMInferenceServiceReady

LLMInferenceService [scheduler-inline-config-test] is Ready

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-7c6c58cc96-rclxb

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-inline-config-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-d9ddfsqtmv

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-d9ddfsqtmv

FailedPreStopHook

PreStopHook failed
(x11)

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-5b7d7d79ff-52bt2

Unhealthy

Startup probe failed: Get "https://10.132.0.61:8000/health": dial tcp 10.132.0.61:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-5b7d7d79ff-52bt2

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-5b7d7d79ff-k79hx

Created

Created container: storage-initializer

kserve-ci-e2e-test

multus

prestop-hook-test-kserve-5b7d7d79ff-k79hx

AddedInterface

Add eth0 [10.132.0.63/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-5b7d7d79ff-k79hx

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:43b18792ad3c8c70fe63e50dee47838dba7fd6b00a2dcfe15c76c1dafc2c7ad8" already present on machine

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-5b7d7d79ff-k79hx

Started

Started container storage-initializer

kserve-ci-e2e-test

replicaset-controller

prestop-hook-test-kserve-5b7d7d79ff

SuccessfulCreate

Created pod: prestop-hook-test-kserve-5b7d7d79ff-k79hx
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-route" not found

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-5b7d7d79ff-k79hx

Started

Started container main

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-5b7d7d79ff-k79hx

Created

Created container: main

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-5b7d7d79ff-k79hx

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test80f99418f0f3eb4c80f02e6144a6d4f2-llmisvc-mode

kserve-ci-e2e-test

deployment-controller

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.Secret kserve-ci-e2e-test/llmisv431d7e30f13554296364403a5b6365f5-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve

kserve-ci-e2e-test

replicaset-controller

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scheduler-79866dc455

SuccessfulCreate

Created pod: llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schezdkxl

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4w2k6k

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4w2k6k

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4w2k6k

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-sa

kserve-ci-e2e-test

deployment-controller

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scheduler-79866dc455 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-rb

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4

SuccessfulCreate

Created pod: llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4w2k6k

kserve-ci-e2e-test

multus

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schezdkxl

AddedInterface

Add eth0 [10.133.0.80/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-workload-svc

kserve-ci-e2e-test

kubelet

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schezdkxl

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schezdkxl

Created

Created container: main

kserve-ci-e2e-test

multus

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4w2k6k

AddedInterface

Add eth0 [10.134.0.30/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schezdkxl

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-5b7d7d79ff-52bt2

Unhealthy

Readiness probe failed: Get "https://10.132.0.61:8000/health": dial tcp 10.132.0.61:8000: connect: connection refused
(x2)

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Updated

Updated v1.Secret kserve-ci-e2e-test/prestop-hook-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv431d7e30f13554296364403a5b6365f5-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-shadow-svc

kserve-ci-e2e-test

endpoint-controller

prestop-hook-test-inference-pool-ip-2912a14d

FailedToUpdateEndpoint

Failed to update endpoint kserve-ci-e2e-test/prestop-hook-test-inference-pool-ip-2912a14d: Operation cannot be fulfilled on endpoints "prestop-hook-test-inference-pool-ip-2912a14d": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/kserve-ci-e2e-test/prestop-hook-test-inference-pool-ip-2912a14d, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 32e064b2-d952-4091-ae3b-d84769ba0c3f, UID in object meta:

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-prestop-hook-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-767d7586b8-gptkj

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-767d7586b8-gptkj

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-5b7d7d79ff-k79hx

Killing

Stopping container main
(x3)

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-5b7d7d79ff-k79hx

Unhealthy

Startup probe failed: Get "https://10.132.0.63:8000/health": dial tcp 10.132.0.63:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-qwen2-5-0-5b-rout-4c1ba212] is Ready

kserve-ci-e2e-test

kubelet

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schezdkxl

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test80f99418f0f3eb4c80f02e6144a6d4f2-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schezdkxl

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4w2k6k

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.Secret kserve-ci-e2e-test/rolling-upgrade-test-kserve-self-signed-certs
(x10)

kserve-ci-e2e-test

OpenDataHubModelController

rolling-upgrade-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy rolling-upgrade-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/rolling-upgrade-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "rolling-upgrade-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.Deployment kserve-ci-e2e-test/rolling-upgrade-test-kserve

kserve-ci-e2e-test

deployment-controller

rolling-upgrade-test-kserve

ScalingReplicaSet

Scaled up replica set rolling-upgrade-test-kserve-54bc76f55b from 0 to 1

kserve-ci-e2e-test

replicaset-controller

rolling-upgrade-test-kserve-54bc76f55b

SuccessfulCreate

Created pod: rolling-upgrade-test-kserve-54bc76f55b-84gdc

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.Service kserve-ci-e2e-test/rolling-upgrade-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-rolling-upgrade-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-54bc76f55b-84gdc

Started

Started container main

kserve-ci-e2e-test

multus

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-hwz6g

AddedInterface

Add eth0 [10.133.0.81/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.Service kserve-ci-e2e-test/rolling-upgrade-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.Deployment kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler

kserve-ci-e2e-test

multus

rolling-upgrade-test-kserve-54bc76f55b-84gdc

AddedInterface

Add eth0 [10.132.0.64/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/rolling-upgrade-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.Role kserve-ci-e2e-test/rolling-upgrade-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/rolling-upgrade-test-epp-sa

kserve-ci-e2e-test

deployment-controller

rolling-upgrade-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd from 0 to 1

kserve-ci-e2e-test

replicaset-controller

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd

SuccessfulCreate

Created pod: rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-hwz6g

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-54bc76f55b-84gdc

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-54bc76f55b-84gdc

Created

Created container: main

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-hwz6g

Created

Created container: main

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-hwz6g

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-hwz6g

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/rolling-upgrade-test-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/rolling-upgrade-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-configmap-ref-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-role
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-configmap-ref-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-configmap-ref-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-configmap-ref-test-kserve-route" not found

kserve-ci-e2e-test

deployment-controller

scheduler-configmap-ref-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-configmap-ref-test-kserve-744bf59fbd from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-workload-svc

kserve-ci-e2e-test

replicaset-controller

scheduler-configmap-ref-test-kserve-744bf59fbd

SuccessfulCreate

Created pod: scheduler-configmap-ref-test-kserve-744bf59fbd-g7knk

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-sa

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-68f87xj26f

Created

Created container: main

kserve-ci-e2e-test

multus

scheduler-configmap-ref-test-kserve-744bf59fbd-g7knk

AddedInterface

Add eth0 [10.134.0.31/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

scheduler-configmap-ref-test-kserve-router-scheduler-68f876cc5d

SuccessfulCreate

Created pod: scheduler-configmap-ref-test-kserve-router-scheduler-68f87xj26f

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-service

kserve-ci-e2e-test

multus

scheduler-configmap-ref-test-kserve-router-scheduler-68f87xj26f

AddedInterface

Add eth0 [10.132.0.65/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-68f87xj26f

Started

Started container main

kserve-ci-e2e-test

deployment-controller

scheduler-configmap-ref-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-configmap-ref-test-kserve-router-scheduler-68f876cc5d from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-rb

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-68f87xj26f

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-744bf59fbd-g7knk

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-744bf59fbd-g7knk

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-744bf59fbd-g7knk

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

LLMInferenceServiceReady

LLMInferenceService [rolling-upgrade-test] is Ready

kserve-ci-e2e-test

deployment-controller

rolling-upgrade-test-kserve

ScalingReplicaSet

Scaled up replica set rolling-upgrade-test-kserve-859df57bcb from 0 to 1

kserve-ci-e2e-test

replicaset-controller

rolling-upgrade-test-kserve-859df57bcb

SuccessfulCreate

Created pod: rolling-upgrade-test-kserve-859df57bcb-7zscs

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Updated

Updated v1.Deployment kserve-ci-e2e-test/rolling-upgrade-test-kserve

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-859df57bcb-7zscs

Started

Started container main

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-859df57bcb-7zscs

Created

Created container: main

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-859df57bcb-7zscs

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

multus

rolling-upgrade-test-kserve-859df57bcb-7zscs

AddedInterface

Add eth0 [10.133.0.82/23] from ovn-kubernetes
(x2)

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Updated

Updated v1.Secret kserve-ci-e2e-test/rolling-upgrade-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-hwz6g

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-54bc76f55b-84gdc

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-hwz6g

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

replicaset-controller

rolling-upgrade-test-kserve-54bc76f55b

SuccessfulDelete

Deleted pod: rolling-upgrade-test-kserve-54bc76f55b-84gdc

kserve-ci-e2e-test

deployment-controller

rolling-upgrade-test-kserve

ScalingReplicaSet

Scaled down replica set rolling-upgrade-test-kserve-54bc76f55b from 1 to 0

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-rolling-upgrade-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-859df57bcb-7zscs

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

LLMInferenceServiceReady

LLMInferenceService [scheduler-configmap-ref-test] is Ready

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-744bf59fbd-g7knk

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-68f87xj26f

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-configmap-ref-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-68f87xj26f

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-54bc76f55b-84gdc

Unhealthy

Readiness probe failed: Get "https://10.132.0.64:8000/health": dial tcp 10.132.0.64:8000: connect: connection refused
(x6)

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-859df57bcb-7zscs

FailedMount

MountVolume.SetUp failed for volume "tls-certs" : secret "rolling-upgrade-test-kserve-self-signed-certs" not found
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-ha-replicas-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-ha-replicas-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-ha-replicas-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve

kserve-ci-e2e-test

replicaset-controller

scheduler-ha-replicas-test-kserve-5c5cfcf776

SuccessfulCreate

Created pod: scheduler-ha-replicas-test-kserve-5c5cfcf776-jlqwh

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs

kserve-ci-e2e-test

deployment-controller

scheduler-ha-replicas-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-ha-replicas-test-kserve-5c5cfcf776 from 0 to 1

kserve-ci-e2e-test

deployment-controller

scheduler-ha-replicas-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dbc4 from 0 to 2

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-ha-replicas-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d9dg95

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-service

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dkvvq9

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dkvvq9

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dkvvq9

Started

Started container main

kserve-ci-e2e-test

replicaset-controller

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dbc4

SuccessfulCreate

Created pod: scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dkvvq9
(x5)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-workload-svc

kserve-ci-e2e-test

multus

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dkvvq9

AddedInterface

Add eth0 [10.134.0.32/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dbc4

SuccessfulCreate

Created pod: scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d9dg95

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d9dg95

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d9dg95

Created

Created container: main

kserve-ci-e2e-test

multus

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d9dg95

AddedInterface

Add eth0 [10.133.0.83/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-5c5cfcf776-jlqwh

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-5c5cfcf776-jlqwh

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-5c5cfcf776-jlqwh

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

multus

scheduler-ha-replicas-test-kserve-5c5cfcf776-jlqwh

AddedInterface

Add eth0 [10.132.0.67/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Error

Reconciliation failed: failed to reconcile workload: failed to reconcile self-signed certificates secret: failed to reconcile self-signed TLS certificate: failed to get defaults for v1.Secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs: context canceled

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.DestinationRule kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

LLMInferenceServiceReady

LLMInferenceService [scheduler-ha-replicas-test] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-ha-replicas-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dkvvq9

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dkvvq9

Killing

Stopping container main
(x4)

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dkvvq9

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d9dg95

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d9dg95

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-5c5cfcf776-jlqwh

Killing

Stopping container main
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-v06-pd-migration-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-v06-pd-migration-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-v06-pd-migration-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-rb

kserve-ci-e2e-test

deployment-controller

scheduler-v06-pd-migration-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-v06-pd-migration-test-kserve-5c87756974 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-role

kserve-ci-e2e-test

deployment-controller

scheduler-v06-pd-migration-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-v06-pd-migration-test-kserve-router-scheduler-59dc679cd from 0 to 1

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-router-scheduler-595ls74

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-pd-migration-test-epp-sa

kserve-ci-e2e-test

multus

scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-dqd46

AddedInterface

Add eth0 [10.134.0.33/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-v06-pd-migration-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-tqwvx

Pulling

Pulling image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2"

kserve-ci-e2e-test

replicaset-controller

scheduler-v06-pd-migration-test-kserve-router-scheduler-59dc679cd

SuccessfulCreate

Created pod: scheduler-v06-pd-migration-test-kserve-router-scheduler-595ls74

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-router-scheduler-595ls74

Started

Started container main

kserve-ci-e2e-test

multus

scheduler-v06-pd-migration-test-kserve-5c87756974-tqwvx

AddedInterface

Add eth0 [10.132.0.68/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-router-scheduler-595ls74

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

multus

scheduler-v06-pd-migration-test-kserve-router-scheduler-595ls74

AddedInterface

Add eth0 [10.133.0.84/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

scheduler-v06-pd-migration-test-kserve-5c87756974

SuccessfulCreate

Created pod: scheduler-v06-pd-migration-test-kserve-5c87756974-tqwvx

kserve-ci-e2e-test

deployment-controller

scheduler-v06-pd-migration-test-kserve-prefill

ScalingReplicaSet

Scaled up replica set scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847

SuccessfulCreate

Created pod: scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-dqd46

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-dqd46

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-dqd46

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-dqd46

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-tqwvx

Created

Created container: llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-tqwvx

Started

Started container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-tqwvx

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" in 725ms (725ms including waiting). Image size: 44914394 bytes.

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-tqwvx

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-tqwvx

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-tqwvx

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine
(x13)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-custom-template-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-custom-template-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-custom-template-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-custom-template-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-custom-template-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-custom-template-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-custom-template-test-epp-sa

kserve-ci-e2e-test

deployment-controller

scheduler-custom-template-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-custom-template-test-kserve-5bc7bd6864 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-custom-template-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-custom-template-test-kserve

kserve-ci-e2e-test

replicaset-controller

scheduler-custom-template-test-kserve-5bc7bd6864

SuccessfulCreate

Created pod: scheduler-custom-template-test-kserve-5bc7bd6864-jbf8p

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-custom-template-test-epp-rb

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-5bc7bd6864-jbf8p

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-custom-template-test-epp-service

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-5bc7bd6864-jbf8p

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

replicaset-controller

scheduler-custom-template-test-kserve-router-scheduler-567469f665

SuccessfulCreate

Created pod: scheduler-custom-template-test-kserve-router-scheduler-5677kqnn

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-custom-template-test-epp-role

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-router-scheduler-5677kqnn

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-router-scheduler-5677kqnn

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-router-scheduler-5677kqnn

Started

Started container main

kserve-ci-e2e-test

multus

scheduler-custom-template-test-kserve-router-scheduler-5677kqnn

AddedInterface

Add eth0 [10.132.0.69/23] from ovn-kubernetes

kserve-ci-e2e-test

multus

scheduler-custom-template-test-kserve-5bc7bd6864-jbf8p

AddedInterface

Add eth0 [10.133.0.85/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

scheduler-custom-template-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-custom-template-test-kserve-router-scheduler-567469f665 from 0 to 1

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-5bc7bd6864-jbf8p

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-route
(x10)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-custom-template-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-custom-template-test-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-custom-template-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

LLMInferenceServiceReady

LLMInferenceService [scheduler-v06-pd-migration-test] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

LLMInferenceServiceReady

LLMInferenceService [scheduler-custom-template-test] is Ready

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-5bc7bd6864-jbf8p

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-router-scheduler-5677kqnn

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-router-scheduler-5677kqnn

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-custom-template-test-epp-auth-rb
(x11)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-v06-threshold-migration-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-v06-threshold-migration-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-v06-threshold-migration-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-workload-svc

kserve-ci-e2e-test

deployment-controller

scheduler-v06-threshold-migration-test-kserve-prefill

ScalingReplicaSet

Scaled up replica set scheduler-v06-threshold-migration-test-kserve-prefill-5fc599dc49 from 0 to 1

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-prefill-5fc5n8sgm

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-prefill-5fc5n8sgm

Created

Created container: main

kserve-ci-e2e-test

replicaset-controller

scheduler-v06-threshold-migration-test-kserve-router-scheduler-dbc65f645

SuccessfulCreate

Created pod: scheduler-v06-threshold-migration-test-kserve-router-sched7z678

kserve-ci-e2e-test

deployment-controller

scheduler-v06-threshold-migration-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-v06-threshold-migration-test-kserve-router-scheduler-dbc65f645 from 0 to 1

kserve-ci-e2e-test

deployment-controller

scheduler-v06-threshold-migration-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-v06-threshold-migration-test-kserve-5994569d5b from 0 to 1

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-prefill-5fc5n8sgm

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

multus

scheduler-v06-threshold-migration-test-kserve-prefill-5fc5n8sgm

AddedInterface

Add eth0 [10.134.0.34/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-role

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve

kserve-ci-e2e-test

replicaset-controller

scheduler-v06-threshold-migration-test-kserve-prefill-5fc599dc49

SuccessfulCreate

Created pod: scheduler-v06-threshold-migration-test-kserve-prefill-5fc5n8sgm

kserve-ci-e2e-test

replicaset-controller

scheduler-v06-threshold-migration-test-kserve-5994569d5b

SuccessfulCreate

Created pod: scheduler-v06-threshold-migration-test-kserve-5994569d5b-cn6jx

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-cn6jx

Started

Started container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-cn6jx

Created

Created container: llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-cn6jx

Pulled

Container image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

multus

scheduler-v06-threshold-migration-test-kserve-5994569d5b-cn6jx

AddedInterface

Add eth0 [10.133.0.86/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test0a07e82be2036793543b2d3db2ea0785-scheduler-v0

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-cn6jx

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-cn6jx

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-cn6jx

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-router-sched7z678

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-router-sched7z678

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-router-sched7z678

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

multus

scheduler-v06-threshold-migration-test-kserve-router-sched7z678

AddedInterface

Add eth0 [10.132.0.70/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-route
(x10)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test0a07e82be2036793543b2d3db2ea0785-scheduler-v0

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

LLMInferenceServiceReady

LLMInferenceService [scheduler-v06-threshold-migration-test] is Ready

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-prefill-5fc5n8sgm

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-cn6jx

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-cn6jx

Killing

Stopping container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-router-sched7z678

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-router-sched7z678

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-cn6jx

Unhealthy

Readiness probe failed: Get "https://10.133.0.86:8001/health": dial tcp 10.133.0.86:8001: connect: connection refused