Time Namespace Component RelatedObject Reason Message

kserve-ci-e2e-test

scheduler-v06-threshold-migration-test-kserve-5994569d5b-sqwkm

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-sqwkm to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79zlqf4

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79zlqf4 to ip-10-0-130-231.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-pd-test-kserve-prefill-d75cf496c-lf5cr

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-d75cf496c-lf5cr to ip-10-0-130-231.ec2.internal

kserve-ci-e2e-test

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4flhjv

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4flhjv to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-ldjxd

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-ldjxd to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

scheduler-v06-threshold-migration-test-kserve-prefill-5fc5st7gp

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5st7gp to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4v8fn5

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4v8fn5 to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-md987

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-md987 to ip-10-0-130-231.ec2.internal

openshift-ingress

router-default-c685cdfc7-h9gsv

Scheduled

Successfully assigned openshift-ingress/router-default-c685cdfc7-h9gsv to ip-10-0-130-231.ec2.internal

kserve-ci-e2e-test

llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd48797p5lt

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd48797p5lt to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

auth-disabled-test-kserve-router-scheduler-7dcc8bd5f8-2tczn

Scheduled

Successfully assigned kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-7dcc8bd5f8-2tczn to ip-10-0-130-231.ec2.internal

kserve-ci-e2e-test

llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9dhnvms

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9dhnvms to ip-10-0-130-231.ec2.internal

kserve-ci-e2e-test

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schesq7lz

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schesq7lz to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche9cdkj

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche9cdkj to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schesfbnj

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schesfbnj to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schesnpfv

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schesnpfv to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schepcq9n

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schepcq9n to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-sche45r6f

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-sche45r6f to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0 to ip-10-0-130-231.ec2.internal

kserve-ci-e2e-test

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schet9l7z

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schet9l7z to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1 to ip-10-0-130-231.ec2.internal

kserve-ci-e2e-test

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheqpzms

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheqpzms to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schew8pjg

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schew8pjg to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schejwtjs

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schejwtjs to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schew2xxx

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schew2xxx to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefgzsz

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefgzsz to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schesbd9d

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schesbd9d to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

lora-multiple-adapters-test-kserve-556cfcc755-lnsk6

Scheduled

Successfully assigned kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-556cfcc755-lnsk6 to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

lora-single-adapter-test-kserve-77b8dc6f6b-qttbp

Scheduled

Successfully assigned kserve-ci-e2e-test/lora-single-adapter-test-kserve-77b8dc6f6b-qttbp to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

precise-prefix-cache-test-kserve-7c6d966575-hf7n8

Scheduled

Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7c6d966575-hf7n8 to ip-10-0-130-231.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-598d446855-8lvfh

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-598d446855-8lvfh to ip-10-0-130-231.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schevwrxh

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schevwrxh to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

auth-disabled-test-kserve-5f858fd8bc-gcn7h

Scheduled

Successfully assigned kserve-ci-e2e-test/auth-disabled-test-kserve-5f858fd8bc-gcn7h to ip-10-0-130-231.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-cf76987fb-h66t4

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-cf76987fb-h66t4 to ip-10-0-130-231.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-pd-test-kserve-db8bc57fd-9h9j7

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-db8bc57fd-9h9j7 to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

auth-enabled-test-kserve-5656f95469-gxppr

Scheduled

Successfully assigned kserve-ci-e2e-test/auth-enabled-test-kserve-5656f95469-gxppr to ip-10-0-130-231.ec2.internal

kserve-ci-e2e-test

precise-prefix-cache-test-kserve-7c6d966575-rtv2x

Scheduled

Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7c6d966575-rtv2x to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4h6l7b

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4h6l7b to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

precise-prefix-cache-test-kserve-router-scheduler-856d559dv8mzv

Scheduled

Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-856d559dv8mzv to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0 to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

prestop-hook-test-kserve-7cc5b79f5-djkbl

Scheduled

Successfully assigned kserve-ci-e2e-test/prestop-hook-test-kserve-7cc5b79f5-djkbl to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

prestop-hook-test-kserve-7cc5b79f5-rwtxg

Scheduled

Successfully assigned kserve-ci-e2e-test/prestop-hook-test-kserve-7cc5b79f5-rwtxg to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

prestop-hook-test-kserve-router-scheduler-577b99f89b-wf8hm

Scheduled

Successfully assigned kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-577b99f89b-wf8hm to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

rolling-upgrade-test-kserve-54bc76f55b-xjr7c

Scheduled

Successfully assigned kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-xjr7c to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

rolling-upgrade-test-kserve-859df57bcb-m6kpq

Scheduled

Successfully assigned kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-m6kpq to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-6kfkp

Scheduled

Successfully assigned kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-6kfkp to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

router-gateway-1-openshift-default-799f46c59b-z2jmj

Scheduled

Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-z2jmj to ip-10-0-130-231.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1 to ip-10-0-130-231.ec2.internal

kserve-ci-e2e-test

auth-enabled-test-kserve-router-scheduler-6c98bb874f-9pkzf

Scheduled

Successfully assigned kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-6c98bb874f-9pkzf to ip-10-0-130-231.ec2.internal

kserve-ci-e2e-test

router-gateway-2-openshift-default-54c789bdc6-dhlsx

Scheduled

Successfully assigned kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-dhlsx to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

router-with-refs-pd-test-kserve-765f8dd694-qtnc6

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-765f8dd694-qtnc6 to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

router-with-refs-pd-test-kserve-prefill-5b6546bb4c-xrkp9

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5b6546bb4c-xrkp9 to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

router-with-refs-pd-test-kserve-router-scheduler-56977fb9ftxcml

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-56977fb9ftxcml to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

router-with-refs-test-kserve-6b55745999-xspsc

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-test-kserve-6b55745999-xspsc to ip-10-0-130-231.ec2.internal

kserve-ci-e2e-test

router-with-refs-test-kserve-router-scheduler-74b964bd8b-zb6dt

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-74b964bd8b-zb6dt to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

scheduler-configmap-ref-test-kserve-744bf59fbd-927xs

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-927xs to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

scheduler-configmap-ref-test-kserve-router-scheduler-68f87gjvtx

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87gjvtx to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

scheduler-custom-template-test-kserve-5bc7bd6864-dm486

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-dm486 to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

scheduler-v06-threshold-migration-test-kserve-router-schedctr8m

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-schedctr8m to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

scheduler-ha-replicas-test-kserve-5c5cfcf776-zt9gz

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-zt9gz to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d7qf6j

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d7qf6j to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dn2c6j

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dn2c6j to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

scheduler-inline-config-test-kserve-7c6c58cc96-kbbbj

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-kbbbj to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

scheduler-inline-config-test-kserve-router-scheduler-d9ddfhgsw6

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfhgsw6 to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

scheduler-v06-pd-migration-test-kserve-5c87756974-pvgch

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-pvgch to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5669d58f5clqbqc

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5669d58f5clqbqc to ip-10-0-130-231.ec2.internal

kserve-ci-e2e-test

scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-dq68c

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-dq68c to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

scheduler-v06-pd-migration-test-kserve-router-scheduler-59kgmqp

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-59kgmqp to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddsx4qz

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddsx4qz to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fgkmx6

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fgkmx6 to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

scheduler-custom-template-test-kserve-router-scheduler-567pxl6w

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567pxl6w to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

stop-feature-test-kserve-7fd76fc4cf-2wz79

Scheduled

Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-7fd76fc4cf-2wz79 to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

stop-feature-test-kserve-7fd76fc4cf-rg7tw

Scheduled

Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-7fd76fc4cf-rg7tw to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

stop-feature-test-kserve-router-scheduler-664b88448d-d7shc

Scheduled

Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-664b88448d-d7shc to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

stop-feature-test-kserve-router-scheduler-664b88448d-hvj9z

Scheduled

Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-664b88448d-hvj9z to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

tls-verification-test-kserve-6bbdc4587f-bc8c7

Scheduled

Successfully assigned kserve-ci-e2e-test/tls-verification-test-kserve-6bbdc4587f-bc8c7 to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

tls-verification-test-kserve-router-scheduler-95f7d4548-8wsbk

Scheduled

Successfully assigned kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-95f7d4548-8wsbk to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

auth-invalid-token-test-kserve-7cffd86bf-pcb7v

Scheduled

Successfully assigned kserve-ci-e2e-test/auth-invalid-token-test-kserve-7cffd86bf-pcb7v to ip-10-0-130-231.ec2.internal

kuadrant-system

authorino-7c77957575-qg2gt

Scheduled

Successfully assigned kuadrant-system/authorino-7c77957575-qg2gt to ip-10-0-139-32.ec2.internal

kuadrant-system

authorino-8df697c9f-nhpvq

Scheduled

Successfully assigned kuadrant-system/authorino-8df697c9f-nhpvq to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-64sg2zn

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-64sg2zn to ip-10-0-130-231.ec2.internal

kuadrant-system

authorino-b6c89bb85-4crp2

Scheduled

Successfully assigned kuadrant-system/authorino-b6c89bb85-4crp2 to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-76cbf745cc72pxw

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-76cbf745cc72pxw to ip-10-0-130-231.ec2.internal

kuadrant-system

authorino-operator-6d85f6564-2rphj

Scheduled

Successfully assigned kuadrant-system/authorino-operator-6d85f6564-2rphj to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-3c960099-kserve-74b7fb866d7p7tf

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-74b7fb866d7p7tf to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bnsnz8

Scheduled

Successfully assigned kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bnsnz8 to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc57f6kd

Scheduled

Successfully assigned kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc57f6kd to ip-10-0-130-231.ec2.internal

kuadrant-system

dns-operator-controller-manager-65b49595d7-bhzx9

Scheduled

Successfully assigned kuadrant-system/dns-operator-controller-manager-65b49595d7-bhzx9 to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

e2e-pvc-model-download-grb75

Scheduled

Successfully assigned kserve-ci-e2e-test/e2e-pvc-model-download-grb75 to ip-10-0-139-32.ec2.internal

kserve-ci-e2e-test

auth-invalid-token-test-kserve-router-scheduler-76b66d78bc5l958

Scheduled

Successfully assigned kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-76b66d78bc5l958 to ip-10-0-130-231.ec2.internal

kuadrant-system

kuadrant-console-plugin-85fd4b8b9d-fmk6t

Scheduled

Successfully assigned kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-fmk6t to ip-10-0-139-32.ec2.internal

kuadrant-system

kuadrant-operator-controller-manager-bb7ddb7d7-n2dpv

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-n2dpv to ip-10-0-139-32.ec2.internal

kuadrant-system

limitador-limitador-69574b596d-8lmfh

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-69574b596d-8lmfh to ip-10-0-139-243.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-test-kserve-router-scheduler-b4dd96dbs59c7

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-b4dd96dbs59c7 to ip-10-0-139-32.ec2.internal

kuadrant-system

limitador-limitador-ff68ff5cb-4mp5j

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-ff68ff5cb-4mp5j to ip-10-0-130-231.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-test-kserve-7458955747-v8trl

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-test-kserve-7458955747-v8trl to ip-10-0-130-231.ec2.internal

kuadrant-system

limitador-operator-controller-manager-6f9f468797-kxmgv

Scheduled

Successfully assigned kuadrant-system/limitador-operator-controller-manager-6f9f468797-kxmgv to ip-10-0-139-32.ec2.internal

openshift-ingress

openshift-ai-inference-openshift-default-9455cf8d5-6qghj

Scheduled

Successfully assigned openshift-ingress/openshift-ai-inference-openshift-default-9455cf8d5-6qghj to ip-10-0-130-231.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-pd-test-kserve-router-scheduler-595d846bdl

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-595d846bdl to ip-10-0-139-32.ec2.internal

openshift-ingress

istiod-openshift-gateway-94bb8fbfd-j9nt7

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j9nt7 to ip-10-0-139-32.ec2.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-c685cdfc7 from 0 to 1
(x10)

openshift-ingress

replicaset-controller

router-default-c685cdfc7

FailedCreate

Error creating: pods "router-default-c685cdfc7-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer
(x16)

openshift-ingress

replicaset-controller

router-default-c685cdfc7

FailedCreate

Error creating: pods "router-default-c685cdfc7-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts

openshift-ingress

replicaset-controller

router-default-c685cdfc7

SuccessfulCreate

Created pod: router-default-c685cdfc7-h9gsv
(x5)

openshift-ingress

kubelet

router-default-c685cdfc7-h9gsv

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x6)

openshift-ingress

kubelet

router-default-c685cdfc7-h9gsv

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt

openshift-ingress

multus

router-default-c685cdfc7-h9gsv

AddedInterface

Add eth0 [10.132.0.10/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-c685cdfc7-h9gsv

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:32d6df0cb6282bc5ba571a8de322fe598afdf22e817f5a7dc92db3274dbdb9a1" already present on machine

openshift-ingress

kubelet

router-default-c685cdfc7-h9gsv

Created

Created container: router

openshift-ingress

kubelet

router-default-c685cdfc7-h9gsv

Started

Started container router

openshift-ingress

replicaset-controller

istiod-openshift-gateway-94bb8fbfd

SuccessfulCreate

Created pod: istiod-openshift-gateway-94bb8fbfd-j9nt7

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-94bb8fbfd from 0 to 1

openshift-ingress

kubelet

istiod-openshift-gateway-94bb8fbfd-j9nt7

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0850242436e88f7d82f0f2126de064c7e0f09844f31d8ff0f53dc8d3908075d9"

openshift-ingress

multus

istiod-openshift-gateway-94bb8fbfd-j9nt7

AddedInterface

Add eth0 [10.133.0.29/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-94bb8fbfd-j9nt7

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-94bb8fbfd-j9nt7

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-94bb8fbfd-j9nt7

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0850242436e88f7d82f0f2126de064c7e0f09844f31d8ff0f53dc8d3908075d9" in 2.847s (2.847s including waiting). Image size: 181034515 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-94bb8fbfd-j9nt7

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-94bb8fbfd-j9nt7

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

openshift-ai-inference-openshift-default

ScalingReplicaSet

Scaled up replica set openshift-ai-inference-openshift-default-9455cf8d5 from 0 to 1

openshift-ingress

gateway_labeler_controller

openshift-ai-inference

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway openshift-ai-inference

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-9455cf8d5-6qghj

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8"

openshift-ingress

multus

openshift-ai-inference-openshift-default-9455cf8d5-6qghj

AddedInterface

Add eth0 [10.132.0.25/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

openshift-ai-inference-openshift-default-9455cf8d5

SuccessfulCreate

Created pod: openshift-ai-inference-openshift-default-9455cf8d5-6qghj

openshift-ingress

service-controller

openshift-ai-inference-openshift-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

openshift-ai-inference-openshift-default

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-9455cf8d5-6qghj

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" in 21.886s (21.886s including waiting). Image size: 181251556 bytes.

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.1

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.1

RequirementsUnknown

requirements not yet checked
(x2)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.1

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-bb7ddb7d7

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-bb7ddb7d7-n2dpv

kuadrant-system

multus

kuadrant-operator-controller-manager-bb7ddb7d7-n2dpv

AddedInterface

Add eth0 [10.133.0.34/23] from ovn-kubernetes

kuadrant-system

kubelet

kuadrant-operator-controller-manager-bb7ddb7d7-n2dpv

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:0b77e4a874b620095a0300877aea53e78559a601ce62b194537ed21c2743bbf3"

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-bb7ddb7d7 from 0 to 1

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.1

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.1

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.1

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.1

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.1

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.0

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

multus

authorino-operator-6d85f6564-2rphj

AddedInterface

Add eth0 [10.134.0.24/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-operator-6d85f6564-2rphj

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:b2cc2c0f0729dad98f970830816e300c0b961ba42491f362295512d886384687"

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.1

InstallWaiting

installing: waiting for deployment authorino-operator to become ready: deployment "authorino-operator" not available: Deployment does not have minimum availability.

kuadrant-system

replicaset-controller

authorino-operator-6d85f6564

SuccessfulCreate

Created pod: authorino-operator-6d85f6564-2rphj

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.1

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

deployment-controller

authorino-operator

ScalingReplicaSet

Scaled up replica set authorino-operator-6d85f6564 from 0 to 1

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.0

RequirementsUnknown

requirements not yet checked

kuadrant-system

kubelet

kuadrant-operator-controller-manager-bb7ddb7d7-n2dpv

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:0b77e4a874b620095a0300877aea53e78559a601ce62b194537ed21c2743bbf3" in 4.072s (4.072s including waiting). Image size: 425661757 bytes.

kuadrant-system

kubelet

kuadrant-operator-controller-manager-bb7ddb7d7-n2dpv

Created

Created container: manager

kuadrant-system

kubelet

kuadrant-operator-controller-manager-bb7ddb7d7-n2dpv

Started

Started container manager

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.0

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

kubelet

authorino-operator-6d85f6564-2rphj

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:b2cc2c0f0729dad98f970830816e300c0b961ba42491f362295512d886384687" in 2.252s (2.252s including waiting). Image size: 168639631 bytes.

kuadrant-system

kuadrant-operator-controller-manager-bb7ddb7d7-n2dpv_a1fd1216-9241-4df0-b45a-bc3b9390ddd8

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-bb7ddb7d7-n2dpv_a1fd1216-9241-4df0-b45a-bc3b9390ddd8 became leader

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.0

InstallWaiting

installing: waiting for deployment limitador-operator-controller-manager to become ready: deployment "limitador-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.0

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.0

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

kubelet

authorino-operator-6d85f6564-2rphj

Created

Created container: manager

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.0

RequirementsUnknown

requirements not yet checked

kuadrant-system

deployment-controller

limitador-operator-controller-manager

ScalingReplicaSet

Scaled up replica set limitador-operator-controller-manager-6f9f468797 from 0 to 1

kuadrant-system

replicaset-controller

limitador-operator-controller-manager-6f9f468797

SuccessfulCreate

Created pod: limitador-operator-controller-manager-6f9f468797-kxmgv

kuadrant-system

kubelet

authorino-operator-6d85f6564-2rphj

Started

Started container manager

kuadrant-system

multus

limitador-operator-controller-manager-6f9f468797-kxmgv

AddedInterface

Add eth0 [10.133.0.35/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-operator-controller-manager-6f9f468797-kxmgv

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:183bad2a6f5872aea38f54ad507f7e005d6facb02b18769e0407c82799f96a3e"

kuadrant-system

authorino-operator-6d85f6564-2rphj_cc9a4d24-bbe9-43fd-8348-829bfc4e3c59

aac3a15d.authorino.kuadrant.io

LeaderElection

authorino-operator-6d85f6564-2rphj_cc9a4d24-bbe9-43fd-8348-829bfc4e3c59 became leader

kuadrant-system

replicaset-controller

dns-operator-controller-manager-65b49595d7

SuccessfulCreate

Created pod: dns-operator-controller-manager-65b49595d7-bhzx9

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.0

InstallWaiting

installing: waiting for deployment dns-operator-controller-manager to become ready: deployment "dns-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.0

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

replicaset-controller

kuadrant-console-plugin-85fd4b8b9d

SuccessfulCreate

Created pod: kuadrant-console-plugin-85fd4b8b9d-fmk6t

kuadrant-system

deployment-controller

kuadrant-console-plugin

ScalingReplicaSet

Scaled up replica set kuadrant-console-plugin-85fd4b8b9d from 0 to 1

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.0

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

deployment-controller

dns-operator-controller-manager

ScalingReplicaSet

Scaled up replica set dns-operator-controller-manager-65b49595d7 from 0 to 1

kuadrant-system

limitador-operator-controller-manager-6f9f468797-kxmgv_b1ab6eea-78f6-4673-8368-b5d58e861840

3745a16e.kuadrant.io

LeaderElection

limitador-operator-controller-manager-6f9f468797-kxmgv_b1ab6eea-78f6-4673-8368-b5d58e861840 became leader

kuadrant-system

kubelet

limitador-operator-controller-manager-6f9f468797-kxmgv

Started

Started container manager

kuadrant-system

kubelet

limitador-operator-controller-manager-6f9f468797-kxmgv

Created

Created container: manager

kuadrant-system

multus

dns-operator-controller-manager-65b49595d7-bhzx9

AddedInterface

Add eth0 [10.133.0.36/23] from ovn-kubernetes

kuadrant-system

kubelet

dns-operator-controller-manager-65b49595d7-bhzx9

Pulling

Pulling image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:ba46f54446ecd3ed1a3d877dcbb7359ff8f2d918827f3411a5cb0781915a65d5"

kuadrant-system

multus

kuadrant-console-plugin-85fd4b8b9d-fmk6t

AddedInterface

Add eth0 [10.133.0.37/23] from ovn-kubernetes

kuadrant-system

kubelet

kuadrant-console-plugin-85fd4b8b9d-fmk6t

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:6e69f41d3d70b0afa7691a5d2400065d3bcdec2ea82830cf2992b8e69ba06e2a"

kuadrant-system

kubelet

limitador-operator-controller-manager-6f9f468797-kxmgv

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:183bad2a6f5872aea38f54ad507f7e005d6facb02b18769e0407c82799f96a3e" in 2.6s (2.6s including waiting). Image size: 178538972 bytes.

kuadrant-system

kubelet

dns-operator-controller-manager-65b49595d7-bhzx9

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:ba46f54446ecd3ed1a3d877dcbb7359ff8f2d918827f3411a5cb0781915a65d5" in 2.113s (2.113s including waiting). Image size: 190472853 bytes.

kuadrant-system

kubelet

dns-operator-controller-manager-65b49595d7-bhzx9

Created

Created container: manager

kuadrant-system

kubelet

dns-operator-controller-manager-65b49595d7-bhzx9

Started

Started container manager

kuadrant-system

dns-operator-controller-manager-65b49595d7-bhzx9_8c0c0287-c173-40eb-aa96-8593053933bf

a3f98d6c.kuadrant.io

LeaderElection

dns-operator-controller-manager-65b49595d7-bhzx9_8c0c0287-c173-40eb-aa96-8593053933bf became leader

kuadrant-system

kubelet

kuadrant-console-plugin-85fd4b8b9d-fmk6t

Created

Created container: kuadrant-console-plugin

kuadrant-system

kubelet

kuadrant-console-plugin-85fd4b8b9d-fmk6t

Started

Started container kuadrant-console-plugin

kuadrant-system

kubelet

kuadrant-console-plugin-85fd4b8b9d-fmk6t

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:6e69f41d3d70b0afa7691a5d2400065d3bcdec2ea82830cf2992b8e69ba06e2a" in 4.677s (4.677s including waiting). Image size: 342502139 bytes.

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.1

InstallSucceeded

install strategy completed with no errors

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.1

InstallSucceeded

install strategy completed with no errors

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.0

InstallSucceeded

install strategy completed with no errors

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.0

InstallSucceeded

install strategy completed with no errors

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-69574b596d from 0 to 1

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-ff68ff5cb from 0 to 1

kuadrant-system

replicaset-controller

limitador-limitador-69574b596d

SuccessfulCreate

Created pod: limitador-limitador-69574b596d-8lmfh

kuadrant-system

replicaset-controller

limitador-limitador-ff68ff5cb

SuccessfulCreate

Created pod: limitador-limitador-ff68ff5cb-4mp5j

kuadrant-system

replicaset-controller

authorino-b6c89bb85

SuccessfulCreate

Created pod: authorino-b6c89bb85-4crp2

kuadrant-system

kubelet

limitador-limitador-69574b596d-8lmfh

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:0b02c000d9a175235223ec5916345a1ccedeedc25c87cc0638e8554e6c318474"

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-b6c89bb85 from 0 to 1

kuadrant-system

kubelet

authorino-b6c89bb85-4crp2

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a"

kuadrant-system

multus

limitador-limitador-69574b596d-8lmfh

AddedInterface

Add eth0 [10.134.0.25/23] from ovn-kubernetes

kuadrant-system

replicaset-controller

authorino-8df697c9f

SuccessfulCreate

Created pod: authorino-8df697c9f-nhpvq

kuadrant-system

multus

limitador-limitador-ff68ff5cb-4mp5j

AddedInterface

Add eth0 [10.132.0.27/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-limitador-ff68ff5cb-4mp5j

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:0b02c000d9a175235223ec5916345a1ccedeedc25c87cc0638e8554e6c318474"

kuadrant-system

multus

authorino-b6c89bb85-4crp2

AddedInterface

Add eth0 [10.133.0.38/23] from ovn-kubernetes

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-8df697c9f from 0 to 1

kuadrant-system

multus

authorino-8df697c9f-nhpvq

AddedInterface

Add eth0 [10.134.0.26/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-8df697c9f-nhpvq

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a"

kuadrant-system

kubelet

limitador-limitador-ff68ff5cb-4mp5j

Created

Created container: limitador

kuadrant-system

kubelet

limitador-limitador-ff68ff5cb-4mp5j

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:0b02c000d9a175235223ec5916345a1ccedeedc25c87cc0638e8554e6c318474" in 3.819s (3.819s including waiting). Image size: 250499765 bytes.

kuadrant-system

kubelet

limitador-limitador-ff68ff5cb-4mp5j

Started

Started container limitador

kuadrant-system

kubelet

authorino-b6c89bb85-4crp2

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a" in 2.298s (2.298s including waiting). Image size: 202706732 bytes.

kuadrant-system

kubelet

authorino-b6c89bb85-4crp2

Created

Created container: authorino

kuadrant-system

kubelet

authorino-b6c89bb85-4crp2

Started

Started container authorino

kuadrant-system

kubelet

limitador-limitador-69574b596d-8lmfh

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:0b02c000d9a175235223ec5916345a1ccedeedc25c87cc0638e8554e6c318474" in 4.572s (4.572s including waiting). Image size: 250499765 bytes.

kuadrant-system

kubelet

limitador-limitador-69574b596d-8lmfh

Created

Created container: limitador

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-b6c89bb85 from 1 to 0

kuadrant-system

kubelet

authorino-8df697c9f-nhpvq

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a" in 3.81s (3.81s including waiting). Image size: 202706732 bytes.

kuadrant-system

kubelet

limitador-limitador-69574b596d-8lmfh

Started

Started container limitador

kuadrant-system

kubelet

authorino-8df697c9f-nhpvq

Created

Created container: authorino

kuadrant-system

kubelet

authorino-8df697c9f-nhpvq

Started

Started container authorino

kuadrant-system

replicaset-controller

authorino-b6c89bb85

SuccessfulDelete

Deleted pod: authorino-b6c89bb85-4crp2

kuadrant-system

kubelet

authorino-b6c89bb85-4crp2

Killing

Stopping container authorino

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled down replica set limitador-limitador-ff68ff5cb from 1 to 0

kuadrant-system

kubelet

limitador-limitador-ff68ff5cb-4mp5j

Killing

Stopping container limitador

kuadrant-system

replicaset-controller

limitador-limitador-ff68ff5cb

SuccessfulDelete

Deleted pod: limitador-limitador-ff68ff5cb-4mp5j

kuadrant-system

replicaset-controller

authorino-7c77957575

SuccessfulCreate

Created pod: authorino-7c77957575-qg2gt

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-7c77957575 from 0 to 1

kuadrant-system

kubelet

authorino-7c77957575-qg2gt

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a"

kuadrant-system

multus

authorino-7c77957575-qg2gt

AddedInterface

Add eth0 [10.133.0.39/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-7c77957575-qg2gt

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a" in 1.099s (1.099s including waiting). Image size: 202706732 bytes.

kuadrant-system

replicaset-controller

authorino-8df697c9f

SuccessfulDelete

Deleted pod: authorino-8df697c9f-nhpvq

kuadrant-system

kubelet

authorino-8df697c9f-nhpvq

Killing

Stopping container authorino

kuadrant-system

kubelet

authorino-7c77957575-qg2gt

Started

Started container authorino

kuadrant-system

kubelet

authorino-7c77957575-qg2gt

Created

Created container: authorino

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-8df697c9f from 1 to 0

kserve-ci-e2e-test

service-controller

router-gateway-1-openshift-default

EnsuringLoadBalancer

Ensuring load balancer

kserve-ci-e2e-test

deployment-controller

router-gateway-1-openshift-default

ScalingReplicaSet

Scaled up replica set router-gateway-1-openshift-default-799f46c59b from 0 to 1

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-799f46c59b

SuccessfulCreate

Created pod: router-gateway-1-openshift-default-799f46c59b-z2jmj

kserve-ci-e2e-test

multus

router-gateway-1-openshift-default-799f46c59b-z2jmj

AddedInterface

Add eth0 [10.132.0.31/23] from ovn-kubernetes

kserve-ci-e2e-test

service-controller

router-gateway-1-openshift-default

EnsuredLoadBalancer

Ensured load balancer

kserve-ci-e2e-test

replicaset-controller

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc5

SuccessfulCreate

Created pod: gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc57f6kd

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.Secret kserve-ci-e2e-test/gw-sec0c69dceeb48768325d1a53a749e65786-kserve-self-signed-certs

kserve-ci-e2e-test

deployment-controller

gw-section-name-router-with-gat-2f0a622e-kserve

ScalingReplicaSet

Scaled up replica set gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc5 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.HTTPRoute kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.Service kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.Deployment kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc57f6kd

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

kserve-ci-e2e-test

multus

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc57f6kd

AddedInterface

Add eth0 [10.132.0.32/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.DestinationRule kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-workload-svc

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc57f6kd

Created

Created container: main

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc57f6kd

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.343s (1.343s including waiting). Image size: 98346788 bytes.

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc57f6kd

Started

Started container main
(x13)

kserve-ci-e2e-test

OpenDataHubModelController

precise-prefix-cache-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy precise-prefix-cache-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/precise-prefix-cache-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "precise-prefix-cache-test-kserve-route" not found

kserve-ci-e2e-test

replicaset-controller

precise-prefix-cache-test-kserve-7c6d966575

SuccessfulCreate

Created pod: precise-prefix-cache-test-kserve-7c6d966575-rtv2x

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Service kserve-ci-e2e-test/precise-prefix-cache-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/precise-prefix-cache-test-epp-sa

kserve-ci-e2e-test

deployment-controller

precise-prefix-cache-test-kserve

ScalingReplicaSet

Scaled up replica set precise-prefix-cache-test-kserve-7c6d966575 from 0 to 2

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-precise-prefix-cache-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Deployment kserve-ci-e2e-test/precise-prefix-cache-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Secret kserve-ci-e2e-test/precise-prefix-cache-test-kserve-self-signed-certs

kserve-ci-e2e-test

replicaset-controller

precise-prefix-cache-test-kserve-7c6d966575

SuccessfulCreate

Created pod: precise-prefix-cache-test-kserve-7c6d966575-hf7n8

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-856d559dv8mzv

Pulling

Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743"

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7c6d966575-rtv2x

Pulling

Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743"

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Service kserve-ci-e2e-test/precise-prefix-cache-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/precise-prefix-cache-test-epp-rb

kserve-ci-e2e-test

multus

precise-prefix-cache-test-kserve-7c6d966575-hf7n8

AddedInterface

Add eth0 [10.132.0.33/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7c6d966575-hf7n8

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7c6d966575-hf7n8

Created

Created container: storage-initializer

kserve-ci-e2e-test

replicaset-controller

precise-prefix-cache-test-kserve-router-scheduler-856d559d5f

SuccessfulCreate

Created pod: precise-prefix-cache-test-kserve-router-scheduler-856d559dv8mzv

kserve-ci-e2e-test

multus

precise-prefix-cache-test-kserve-7c6d966575-rtv2x

AddedInterface

Add eth0 [10.133.0.41/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7c6d966575-hf7n8

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Deployment kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler

kserve-ci-e2e-test

multus

precise-prefix-cache-test-kserve-router-scheduler-856d559dv8mzv

AddedInterface

Add eth0 [10.133.0.42/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Role kserve-ci-e2e-test/precise-prefix-cache-test-epp-role

kserve-ci-e2e-test

deployment-controller

precise-prefix-cache-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set precise-prefix-cache-test-kserve-router-scheduler-856d559d5f from 0 to 1
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-z2jmj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-z2jmj

Created

Created container: istio-proxy
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-z2jmj

Started

Started container istio-proxy

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-z2jmj

Unhealthy

Readiness probe failed: Get "http://10.132.0.31:15021/healthz/ready": read tcp 10.132.0.2:40350->10.132.0.31:15021: read: connection reset by peer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-856d559dv8mzv

Pulled

Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" in 3.286s (3.286s including waiting). Image size: 300570045 bytes.

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7c6d966575-rtv2x

Pulled

Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" in 3.63s (3.63s including waiting). Image size: 300570045 bytes.

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-856d559dv8mzv

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7c6d966575-rtv2x

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7c6d966575-rtv2x

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc57f6kd

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-856d559dv8mzv

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7c6d966575-hf7n8

Started

Started container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7c6d966575-hf7n8

Created

Created container: main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-856d559dv8mzv

Pulling

Pulling image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2"

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7c6d966575-hf7n8

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-856d559dv8mzv

Pulling

Pulling image "ghcr.io/llm-d/llm-d-uds-tokenizer:vllm-v0.19.1"

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-856d559dv8mzv

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" in 1.322s (1.322s including waiting). Image size: 75073927 bytes.

kserve-ci-e2e-test

deployment-controller

gw-section-name-router-with-gat-f1d92d0f-kserve

ScalingReplicaSet

Scaled up replica set gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.HTTPRoute kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.DestinationRule kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.Secret kserve-ci-e2e-test/gw-sec2774c263d49959f50d9eebc552e13bf9-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.Deployment kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve

kserve-ci-e2e-test

replicaset-controller

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b

SuccessfulCreate

Created pod: gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bnsnz8
(x5)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-z2jmj

Unhealthy

Startup probe failed: Get "http://10.132.0.31:15021/healthz/ready": dial tcp 10.132.0.31:15021: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.Service kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-workload-svc

kserve-ci-e2e-test

multus

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bnsnz8

AddedInterface

Add eth0 [10.134.0.27/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7c6d966575-rtv2x

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bnsnz8

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7c6d966575-rtv2x

Started

Started container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7c6d966575-rtv2x

Created

Created container: main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7c6d966575-rtv2x

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.289s (1.289s including waiting). Image size: 98346788 bytes.

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bnsnz8

Started

Started container main

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bnsnz8

Created

Created container: main

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bnsnz8

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 968ms (969ms including waiting). Image size: 98346788 bytes.
(x6)

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc57f6kd

FailedMount

MountVolume.SetUp failed for volume "tls-certs" : secret "gw-sec0c69dceeb48768325d1a53a749e65786-kserve-self-signed-certs" not found

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/precise-prefix-cache-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Updated

Updated v1.Secret kserve-ci-e2e-test/precise-prefix-cache-test-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/precise-prefix-cache-test-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bnsnz8

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-856d559dv8mzv

Created

Created container: tokenizer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-856d559dv8mzv

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-856d559dv8mzv

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-uds-tokenizer:vllm-v0.19.1" in 30.883s (30.883s including waiting). Image size: 2989890188 bytes.
(x2)

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-856d559dv8mzv

Created

Created container: main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-856d559dv8mzv

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine
(x2)

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-856d559dv8mzv

Started

Started container main

kserve-ci-e2e-test

replicaset-controller

auth-enabled-test-kserve-5656f95469

SuccessfulCreate

Created pod: auth-enabled-test-kserve-5656f95469-gxppr

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.Deployment kserve-ci-e2e-test/auth-enabled-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/auth-enabled-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.Role kserve-ci-e2e-test/auth-enabled-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-enabled-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bnsnz8

Unhealthy

Readiness probe failed: Get "https://10.134.0.27:8000/health": context deadline exceeded

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/auth-enabled-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.Service kserve-ci-e2e-test/auth-enabled-test-kserve-workload-svc

kserve-ci-e2e-test

deployment-controller

auth-enabled-test-kserve

ScalingReplicaSet

Scaled up replica set auth-enabled-test-kserve-5656f95469 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.Secret kserve-ci-e2e-test/auth-enabled-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-6c98bb874f-9pkzf

Created

Created container: storage-initializer

kserve-ci-e2e-test

deployment-controller

auth-enabled-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set auth-enabled-test-kserve-router-scheduler-6c98bb874f from 0 to 1

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-6c98bb874f-9pkzf

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-6c98bb874f-9pkzf

Started

Started container storage-initializer

kserve-ci-e2e-test

replicaset-controller

auth-enabled-test-kserve-router-scheduler-6c98bb874f

SuccessfulCreate

Created pod: auth-enabled-test-kserve-router-scheduler-6c98bb874f-9pkzf

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-5656f95469-gxppr

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.Deployment kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.Service kserve-ci-e2e-test/auth-enabled-test-epp-service

kserve-ci-e2e-test

multus

auth-enabled-test-kserve-5656f95469-gxppr

AddedInterface

Add eth0 [10.132.0.34/23] from ovn-kubernetes

kserve-ci-e2e-test

multus

auth-enabled-test-kserve-router-scheduler-6c98bb874f-9pkzf

AddedInterface

Add eth0 [10.132.0.35/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-5656f95469-gxppr

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-5656f95469-gxppr

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-6c98bb874f-9pkzf

Pulling

Pulling image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2"

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-6c98bb874f-9pkzf

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" in 1.337s (1.337s including waiting). Image size: 75073927 bytes.

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-856d559dv8mzv

Unhealthy

Liveness probe failed: timeout: failed to connect service "10.133.0.42:9003" within 1s: context deadline exceeded

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-6c98bb874f-9pkzf

Created

Created container: main

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-6c98bb874f-9pkzf

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Updated

Updated v1.Secret kserve-ci-e2e-test/auth-enabled-test-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/auth-enabled-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/auth-enabled-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

LLMInferenceServiceReady

LLMInferenceService [precise-prefix-cache-test] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-precise-prefix-cache-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7c6d966575-hf7n8

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-7c6d966575-rtv2x

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-856d559dv8mzv

Killing

Stopping container tokenizer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-856d559dv8mzv

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-856d559dv8mzv

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-5656f95469-gxppr

Pulling

Pulling image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0"
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-router-managed-test-llm-e45d1f79

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-router-managed-test-llm-e45d1f79-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-router-managed-test-llm-e45d1f79-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.Secret kserve-ci-e2e-test/llmisv5c7e67b6c51568d1d6d13829a9337f2a-kserve-self-signed-certs

kserve-ci-e2e-test

multus

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-sche45r6f

AddedInterface

Add eth0 [10.133.0.43/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheduler-5dd88bfbb7

SuccessfulCreate

Created pod: llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-sche45r6f

kserve-ci-e2e-test

kubelet

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-sche45r6f

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-sche45r6f

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-sche45r6f

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-testef4d2875be14b30dc1561ed84d0d4bde-llmisvc-rout

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-rb

kserve-ci-e2e-test

deployment-controller

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheduler-5dd88bfbb7 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-sa

kserve-ci-e2e-test

multus

llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd48797p5lt

AddedInterface

Add eth0 [10.134.0.28/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd48797p5lt

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd48797p5lt

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve

kserve-ci-e2e-test

replicaset-controller

llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879

SuccessfulCreate

Created pod: llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd48797p5lt

kserve-ci-e2e-test

deployment-controller

llmisvc-router-managed-test-llm-e45d1f79-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879 from 0 to 1

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd48797p5lt

Started

Started container main

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-5656f95469-gxppr

Pulled

Successfully pulled image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" in 27.394s (27.394s including waiting). Image size: 3531177328 bytes.

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-5656f95469-gxppr

Started

Started container main

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-5656f95469-gxppr

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv5c7e67b6c51568d1d6d13829a9337f2a-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

LLMInferenceServiceReady

LLMInferenceService [llmisvc-router-managed-test-llm-e45d1f79] is Ready

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd48797p5lt

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-sche45r6f

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-sche45r6f

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-testef4d2875be14b30dc1561ed84d0d4bde-llmisvc-rout
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-router-managed-test-llm-5b1e8f15

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-router-managed-test-llm-5b1e8f15-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-router-managed-test-llm-5b1e8f15-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve

kserve-ci-e2e-test

replicaset-controller

llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4

SuccessfulCreate

Created pod: llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4v8fn5

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheduler

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4v8fn5

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-rb

kserve-ci-e2e-test

replicaset-controller

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheduler-749449dbc8

SuccessfulCreate

Created pod: llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schesbd9d

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test7f54e84970003a6e7372bdbcb574f7ed-llmisvc-rout

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-workload-svc

kserve-ci-e2e-test

deployment-controller

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheduler-749449dbc8 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.Secret kserve-ci-e2e-test/llmisve55ae740357a3a31a27cdb8b66ffe20f-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4v8fn5

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4v8fn5

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

multus

llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4v8fn5

AddedInterface

Add eth0 [10.134.0.29/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-service

kserve-ci-e2e-test

deployment-controller

llmisvc-router-managed-test-llm-5b1e8f15-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4 from 0 to 1

kserve-ci-e2e-test

multus

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schesbd9d

AddedInterface

Add eth0 [10.133.0.44/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schesbd9d

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schesbd9d

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schesbd9d

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisve55ae740357a3a31a27cdb8b66ffe20f-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

LLMInferenceServiceReady

LLMInferenceService [llmisvc-router-managed-test-llm-5b1e8f15] is Ready
(x11)

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-5656f95469-gxppr

Unhealthy

Startup probe failed: Get "https://10.132.0.34:8000/health": dial tcp 10.132.0.34:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

LLMInferenceServiceReady

LLMInferenceService [auth-enabled-test] is Ready

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-6c98bb874f-9pkzf

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-5656f95469-gxppr

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-enabled-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-6c98bb874f-9pkzf

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-6c98bb874f-9pkzf

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.132.0.35:9003" within 1s: context deadline exceeded

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-5656f95469-gxppr

Unhealthy

Readiness probe failed: Get "https://10.132.0.34:8000/health": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

kserve-ci-e2e-test

deployment-controller

auth-invalid-token-test-kserve

ScalingReplicaSet

Scaled up replica set auth-invalid-token-test-kserve-7cffd86bf from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/auth-invalid-token-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/auth-invalid-token-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.Deployment kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.Service kserve-ci-e2e-test/auth-invalid-token-test-epp-service

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-7cffd86bf-pcb7v

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-7cffd86bf-pcb7v

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-7cffd86bf-pcb7v

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-invalid-token-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.Role kserve-ci-e2e-test/auth-invalid-token-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.Service kserve-ci-e2e-test/auth-invalid-token-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.Deployment kserve-ci-e2e-test/auth-invalid-token-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.Secret kserve-ci-e2e-test/auth-invalid-token-test-kserve-self-signed-certs

kserve-ci-e2e-test

deployment-controller

auth-invalid-token-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set auth-invalid-token-test-kserve-router-scheduler-76b66d78bc from 0 to 1

kserve-ci-e2e-test

replicaset-controller

auth-invalid-token-test-kserve-7cffd86bf

SuccessfulCreate

Created pod: auth-invalid-token-test-kserve-7cffd86bf-pcb7v

kserve-ci-e2e-test

replicaset-controller

auth-invalid-token-test-kserve-router-scheduler-76b66d78bc

SuccessfulCreate

Created pod: auth-invalid-token-test-kserve-router-scheduler-76b66d78bc5l958

kserve-ci-e2e-test

multus

auth-invalid-token-test-kserve-7cffd86bf-pcb7v

AddedInterface

Add eth0 [10.132.0.36/23] from ovn-kubernetes

kserve-ci-e2e-test

multus

auth-invalid-token-test-kserve-router-scheduler-76b66d78bc5l958

AddedInterface

Add eth0 [10.132.0.37/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-76b66d78bc5l958

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-76b66d78bc5l958

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-76b66d78bc5l958

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-76b66d78bc5l958

Started

Started container main

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-76b66d78bc5l958

Created

Created container: main

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-76b66d78bc5l958

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-7cffd86bf-pcb7v

Started

Started container main

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-7cffd86bf-pcb7v

Created

Created container: main

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-7cffd86bf-pcb7v

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/auth-invalid-token-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Updated

Updated v1.Secret kserve-ci-e2e-test/auth-invalid-token-test-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/auth-invalid-token-test-kserve-shadow-svc
(x10)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-sa

kserve-ci-e2e-test

deployment-controller

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58dd from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-testb7025ca4d8a6f8f5b2fd08b5581d2678-llmisvc-mode

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-service

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58dd

SuccessfulCreate

Created pod: llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddsx4qz

kserve-ci-e2e-test

multus

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddsx4qz

AddedInterface

Add eth0 [10.134.0.30/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheduler-6d86bd4d9d from 0 to 1

kserve-ci-e2e-test

replicaset-controller

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheduler-6d86bd4d9d

SuccessfulCreate

Created pod: llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schew8pjg

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddsx4qz

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddsx4qz

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddsx4qz

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.Secret kserve-ci-e2e-test/llmisvb11a9c9832b99b016bc8f8e0ea095712-kserve-self-signed-certs

kserve-ci-e2e-test

multus

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schew8pjg

AddedInterface

Add eth0 [10.133.0.45/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schew8pjg

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schew8pjg

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schew8pjg

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisvb11a9c9832b99b016bc8f8e0ea095712-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-qwen2-5-0-5b-rout-a50492e9] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-testb7025ca4d8a6f8f5b2fd08b5581d2678-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddsx4qz

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schew8pjg

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schew8pjg

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddsx4qz

Unhealthy

Readiness probe failed: Get "https://10.134.0.30:8000/health": EOF

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4v8fn5

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schesbd9d

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schesbd9d

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test7f54e84970003a6e7372bdbcb574f7ed-llmisvc-rout
(x11)

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-7cffd86bf-pcb7v

Unhealthy

Startup probe failed: Get "https://10.132.0.36:8000/health": dial tcp 10.132.0.36:8000: connect: connection refused
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-router-managed-test-llm-4b931143

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-router-managed-test-llm-4b931143-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-router-managed-test-llm-4b931143-kserve-route" not found

kserve-ci-e2e-test

deployment-controller

llmisvc-router-managed-test-llm-4b931143-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56

SuccessfulCreate

Created pod: llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-ldjxd

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test8ac8e3d2264ccb939eb021b0b835847c-llmisvc-rout

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.Secret kserve-ci-e2e-test/llmisvca2d2d7d499abb359505529ebe02c136-kserve-self-signed-certs

kserve-ci-e2e-test

multus

llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-ldjxd

AddedInterface

Add eth0 [10.134.0.31/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-scheduler-68cc9685d6

SuccessfulCreate

Created pod: llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schew2xxx

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-service

kserve-ci-e2e-test

kubelet

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schew2xxx

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schew2xxx

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schew2xxx

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

multus

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schew2xxx

AddedInterface

Add eth0 [10.133.0.46/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-ldjxd

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-ldjxd

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-ldjxd

Started

Started container main

kserve-ci-e2e-test

deployment-controller

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-scheduler-68cc9685d6 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

LLMInferenceServiceReady

LLMInferenceService [auth-invalid-token-test] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisvca2d2d7d499abb359505529ebe02c136-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-invalid-token-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-76b66d78bc5l958

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-76b66d78bc5l958

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-7cffd86bf-pcb7v

Killing

Stopping container main
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

auth-disabled-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy auth-disabled-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/auth-disabled-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "auth-disabled-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/auth-disabled-test-epp-rb

kserve-ci-e2e-test

multus

auth-disabled-test-kserve-5f858fd8bc-gcn7h

AddedInterface

Add eth0 [10.132.0.38/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-5f858fd8bc-gcn7h

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-router-scheduler-7dcc8bd5f8-2tczn

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-router-scheduler-7dcc8bd5f8-2tczn

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.Service kserve-ci-e2e-test/auth-disabled-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.Deployment kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.Role kserve-ci-e2e-test/auth-disabled-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-disabled-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/auth-disabled-test-epp-sa

kserve-ci-e2e-test

multus

auth-disabled-test-kserve-router-scheduler-7dcc8bd5f8-2tczn

AddedInterface

Add eth0 [10.132.0.39/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-5f858fd8bc-gcn7h

Created

Created container: storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.Service kserve-ci-e2e-test/auth-disabled-test-kserve-workload-svc

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-5f858fd8bc-gcn7h

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.Deployment kserve-ci-e2e-test/auth-disabled-test-kserve

kserve-ci-e2e-test

replicaset-controller

auth-disabled-test-kserve-router-scheduler-7dcc8bd5f8

SuccessfulCreate

Created pod: auth-disabled-test-kserve-router-scheduler-7dcc8bd5f8-2tczn

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.Secret kserve-ci-e2e-test/auth-disabled-test-kserve-self-signed-certs

kserve-ci-e2e-test

deployment-controller

auth-disabled-test-kserve

ScalingReplicaSet

Scaled up replica set auth-disabled-test-kserve-5f858fd8bc from 0 to 1

kserve-ci-e2e-test

deployment-controller

auth-disabled-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set auth-disabled-test-kserve-router-scheduler-7dcc8bd5f8 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

auth-disabled-test-kserve-5f858fd8bc

SuccessfulCreate

Created pod: auth-disabled-test-kserve-5f858fd8bc-gcn7h

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-router-scheduler-7dcc8bd5f8-2tczn

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-router-scheduler-7dcc8bd5f8-2tczn

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

LLMInferenceServiceReady

LLMInferenceService [llmisvc-router-managed-test-llm-4b931143] is Ready

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-router-scheduler-7dcc8bd5f8-2tczn

Created

Created container: main

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-router-scheduler-7dcc8bd5f8-2tczn

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Updated

Updated v1.Secret kserve-ci-e2e-test/auth-disabled-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/auth-disabled-test-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/auth-disabled-test-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-5f858fd8bc-gcn7h

Created

Created container: main

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-5f858fd8bc-gcn7h

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-5f858fd8bc-gcn7h

Started

Started container main

kserve-ci-e2e-test

deployment-controller

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheduler-6fcb489785 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.Secret kserve-ci-e2e-test/llmisv08544b88a8d963ffd553cc1f3ed82d16-kserve-self-signed-certs
(x13)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-service

kserve-ci-e2e-test

deployment-controller

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheduler

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f

SuccessfulCreate

Created pod: llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fgkmx6

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test76d7f532acb694e4a7bcef75d32cd8a1-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schesq7lz

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fgkmx6

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schesq7lz

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

multus

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schesq7lz

AddedInterface

Add eth0 [10.133.0.47/23] from ovn-kubernetes

kserve-ci-e2e-test

multus

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fgkmx6

AddedInterface

Add eth0 [10.134.0.32/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schesq7lz

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fgkmx6

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fgkmx6

Started

Started container main

kserve-ci-e2e-test

replicaset-controller

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheduler-6fcb489785

SuccessfulCreate

Created pod: llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schesq7lz

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv08544b88a8d963ffd553cc1f3ed82d16-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-qwen2-5-0-5b-rout-4f8c0978] is Ready

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fgkmx6

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test76d7f532acb694e4a7bcef75d32cd8a1-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schesq7lz

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schesq7lz

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schew2xxx

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test8ac8e3d2264ccb939eb021b0b835847c-llmisvc-rout

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-ldjxd

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schew2xxx

Killing

Stopping container main
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-with-7ca60146

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route" not found

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheduler-766bf6cdf7

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schevwrxh

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-service

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-with-7ca60146-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-with-7ca60146-kserve-cf76987fb from 0 to 1

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheduler-766bf6cdf7 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test5216bfd716f919dc046bc693ceb22e41-llmisvc-mode

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.Secret kserve-ci-e2e-test/llmisv3e414c2ba058a022dfd694dbcbac5b51-kserve-self-signed-certs

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-cf76987fb

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-with-7ca60146-kserve-cf76987fb-h66t4

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheduler

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schevwrxh

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schevwrxh

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schevwrxh

Started

Started container storage-initializer

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-cf76987fb-h66t4

AddedInterface

Add eth0 [10.132.0.40/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-cf76987fb-h66t4

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-cf76987fb-h66t4

Started

Started container storage-initializer

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schevwrxh

AddedInterface

Add eth0 [10.133.0.48/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-cf76987fb-h66t4

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schevwrxh

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schevwrxh

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schevwrxh

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-cf76987fb-h66t4

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-cf76987fb-h66t4

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-cf76987fb-h66t4

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv3e414c2ba058a022dfd694dbcbac5b51-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-shadow-svc
(x11)

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-5f858fd8bc-gcn7h

Unhealthy

Startup probe failed: Get "https://10.132.0.38:8000/health": dial tcp 10.132.0.38:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

LLMInferenceServiceReady

LLMInferenceService [auth-disabled-test] is Ready
(x11)

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-9455cf8d5-6qghj

Unhealthy

Startup probe failed: Get "http://10.132.0.25:15021/healthz/ready": dial tcp 10.132.0.25:15021: connect: connection refused

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-9455cf8d5-6qghj

Unhealthy

Startup probe failed: Get "http://10.132.0.25:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-9455cf8d5-6qghj

ProbeError

Startup probe error: Get "http://10.132.0.25:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x8)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-cf76987fb-h66t4

Unhealthy

Startup probe failed: Get "https://10.132.0.40:8000/health": dial tcp 10.132.0.40:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-fb-opt-125m-with-7ca60146] is Ready
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-e95b1dc1

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-e95b1dc1

Created

Created v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-route

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4 from 0 to 1

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4h6l7b

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4h6l7b

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4h6l7b

AddedInterface

Add eth0 [10.134.0.33/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4h6l7b

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-e95b1dc1

Created

Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-workload-svc

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4h6l7b

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-e95b1dc1

Created

Created v1.Secret kserve-ci-e2e-test/llmisv122f03714c5bdf915a2917fdf1262b98-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-e95b1dc1

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-e95b1dc1

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-e95b1dc1

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-fb-opt-125m-route-e95b1dc1] is Ready

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4h6l7b

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-e95b1dc1

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv122f03714c5bdf915a2917fdf1262b98-kserve-self-signed-certs
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-87882a8e

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-87882a8e-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-87882a8e-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-workload-svc

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5669d58f5clqbqc

Created

Created container: storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.Secret kserve-ci-e2e-test/llmisva690bbc929faec8bc98c767f16c003c1-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheduler

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-87882a8e-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5669d58f5c from 0 to 1

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5669d58f5c

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5669d58f5clqbqc

kserve-ci-e2e-test

replicaset-controller

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheduler-556f758b5f

SuccessfulCreate

Created pod: llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheqpzms

kserve-ci-e2e-test

deployment-controller

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheduler-556f758b5f from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test21fe6730fe484f3a92b1a16afe1bac8f-llmisvc-mode

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5669d58f5clqbqc

AddedInterface

Add eth0 [10.132.0.41/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5669d58f5clqbqc

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5669d58f5clqbqc

Started

Started container storage-initializer

kserve-ci-e2e-test

multus

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheqpzms

AddedInterface

Add eth0 [10.133.0.49/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheqpzms

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheqpzms

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheqpzms

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheqpzms

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheqpzms

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheqpzms

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5669d58f5clqbqc

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5669d58f5clqbqc

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5669d58f5clqbqc

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisva690bbc929faec8bc98c767f16c003c1-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-route
(x12)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5669d58f5clqbqc

Unhealthy

Startup probe failed: Get "https://10.132.0.41:8000/health": dial tcp 10.132.0.41:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-fb-opt-125m-route-87882a8e] is Ready

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5669d58f5clqbqc

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test21fe6730fe484f3a92b1a16afe1bac8f-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheqpzms

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheqpzms

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5669d58f5clqbqc

Unhealthy

Readiness probe failed: Get "https://10.132.0.41:8000/health": dial tcp 10.132.0.41:8000: connect: connection refused
(x11)

kserve-ci-e2e-test

OpenDataHubModelController

custom-route-timeout-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy custom-route-timeout-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/custom-route-timeout-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "custom-route-timeout-test-kserve-route" not found

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-test-kserve

ScalingReplicaSet

Scaled up replica set custom-route-timeout-test-kserve-7458955747 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-test-kserve-7458955747

SuccessfulCreate

Created pod: custom-route-timeout-test-kserve-7458955747-v8trl

kserve-ci-e2e-test

multus

custom-route-timeout-test-kserve-7458955747-v8trl

AddedInterface

Add eth0 [10.132.0.42/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-7458955747-v8trl

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-7458955747-v8trl

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-7458955747-v8trl

Started

Started container storage-initializer

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-test-kserve-router-scheduler-b4dd96db

SuccessfulCreate

Created pod: custom-route-timeout-test-kserve-router-scheduler-b4dd96dbs59c7

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set custom-route-timeout-test-kserve-router-scheduler-b4dd96db from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Secret kserve-ci-e2e-test/custom-route-timeout-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Service kserve-ci-e2e-test/custom-route-timeout-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Role kserve-ci-e2e-test/custom-route-timeout-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/custom-route-timeout-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Service kserve-ci-e2e-test/custom-route-timeout-test-epp-service

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-b4dd96dbs59c7

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-b4dd96dbs59c7

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-b4dd96dbs59c7

Created

Created container: storage-initializer

kserve-ci-e2e-test

multus

custom-route-timeout-test-kserve-router-scheduler-b4dd96dbs59c7

AddedInterface

Add eth0 [10.133.0.50/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-b4dd96dbs59c7

Created

Created container: main

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-b4dd96dbs59c7

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-b4dd96dbs59c7

Started

Started container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-7458955747-v8trl

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-7458955747-v8trl

Started

Started container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-7458955747-v8trl

Created

Created container: main
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/custom-route-timeout-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Updated

Updated v1.Secret kserve-ci-e2e-test/custom-route-timeout-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/custom-route-timeout-test-kserve-route
(x11)

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-7458955747-v8trl

Unhealthy

Startup probe failed: Get "https://10.132.0.42:8000/health": dial tcp 10.132.0.42:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

LLMInferenceServiceReady

LLMInferenceService [custom-route-timeout-test] is Ready
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-with-ba4d693a

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route" not found

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-scheduler-67cc76c46d

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefgzsz

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test05addb65ba05195619f26ef266e8fc04-llmisvc-mode

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.Secret kserve-ci-e2e-test/llmisv77ff2528d3e9b4972cd9335229fce9f0-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-rb

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-598d446855

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-598d446855-8lvfh

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-workload-svc

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-598d446855 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-service

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-scheduler-67cc76c46d from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-sa

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefgzsz

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-598d446855-8lvfh

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-598d446855-8lvfh

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefgzsz

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefgzsz

Created

Created container: storage-initializer

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-598d446855-8lvfh

AddedInterface

Add eth0 [10.132.0.43/23] from ovn-kubernetes

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefgzsz

AddedInterface

Add eth0 [10.133.0.51/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-598d446855-8lvfh

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefgzsz

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefgzsz

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefgzsz

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-598d446855-8lvfh

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-598d446855-8lvfh

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-598d446855-8lvfh

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv77ff2528d3e9b4972cd9335229fce9f0-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-shadow-svc
(x7)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-598d446855-8lvfh

Unhealthy

Startup probe failed: Get "https://10.132.0.43:8000/health": dial tcp 10.132.0.43:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-fb-opt-125m-with-ba4d693a] is Ready
(x10)

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-9455cf8d5-6qghj

Started

Started container istio-proxy

kserve-ci-e2e-test

job-controller

e2e-pvc-model-download

SuccessfulCreate

Created pod: e2e-pvc-model-download-grb75

kserve-ci-e2e-test

persistentvolume-controller

e2e-pvc-model-storage

WaitForFirstConsumer

waiting for first consumer to be created before binding

kserve-ci-e2e-test

ebs.csi.aws.com_aws-ebs-csi-driver-controller-544b56d47d-xltdm_405efd3a-3a6b-4ff2-9d06-722a9c9af554

e2e-pvc-model-storage

Provisioning

External provisioner is provisioning volume for claim "kserve-ci-e2e-test/e2e-pvc-model-storage"
(x2)

kserve-ci-e2e-test

persistentvolume-controller

e2e-pvc-model-storage

ExternalProvisioning

Waiting for a volume to be created either by the external provisioner 'ebs.csi.aws.com' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered.

kserve-ci-e2e-test

ebs.csi.aws.com_aws-ebs-csi-driver-controller-544b56d47d-xltdm_405efd3a-3a6b-4ff2-9d06-722a9c9af554

e2e-pvc-model-storage

ProvisioningSucceeded

Successfully provisioned volume pvc-c58d8f9f-3948-479b-bcc9-16202a5f1201

kserve-ci-e2e-test

attachdetach-controller

e2e-pvc-model-download-grb75

SuccessfulAttachVolume

AttachVolume.Attach succeeded for volume "pvc-c58d8f9f-3948-479b-bcc9-16202a5f1201"

kserve-ci-e2e-test

kubelet

e2e-pvc-model-download-grb75

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

e2e-pvc-model-download-grb75

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

e2e-pvc-model-download-grb75

Started

Started container storage-initializer

kserve-ci-e2e-test

multus

e2e-pvc-model-download-grb75

AddedInterface

Add eth0 [10.133.0.52/23] from ovn-kubernetes

kserve-ci-e2e-test

job-controller

e2e-pvc-model-download

Completed

Job completed
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-pvc-router-manage-59b9d263

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-pvc-router-manage-59b9d263-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-pvc-router-manage-59b9d263-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.Secret kserve-ci-e2e-test/llmisv8bf079eb6eda4debfb4ef5bb7817824c-kserve-self-signed-certs

kserve-ci-e2e-test

deployment-controller

llmisvc-model-pvc-router-manage-59b9d263-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9d from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9d

SuccessfulCreate

Created pod: llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9dhnvms

kserve-ci-e2e-test

deployment-controller

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheduler-5f555d4d85 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheduler-5f555d4d85

SuccessfulCreate

Created pod: llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schet9l7z

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-epp-service

kserve-ci-e2e-test

multus

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schet9l7z

AddedInterface

Add eth0 [10.134.0.34/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schet9l7z

Pulling

Pulling image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2"

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-testc9569cf4801efc0ed27b2f25ffaee875-llmisvc-mode

kserve-ci-e2e-test

attachdetach-controller

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schet9l7z

FailedAttachVolume

Multi-Attach error for volume "pvc-c58d8f9f-3948-479b-bcc9-16202a5f1201" Volume is already used by pod(s) llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9dhnvms

kserve-ci-e2e-test

kubelet

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schet9l7z

Started

Started container main

kserve-ci-e2e-test

attachdetach-controller

llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9dhnvms

SuccessfulAttachVolume

AttachVolume.Attach succeeded for volume "pvc-c58d8f9f-3948-479b-bcc9-16202a5f1201"

kserve-ci-e2e-test

kubelet

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schet9l7z

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" in 1.643s (1.643s including waiting). Image size: 75073927 bytes.

kserve-ci-e2e-test

kubelet

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schet9l7z

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9dhnvms

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9dhnvms

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

multus

llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9dhnvms

AddedInterface

Add eth0 [10.132.0.44/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9dhnvms

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-shadow-svc
(x2)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv8bf079eb6eda4debfb4ef5bb7817824c-kserve-self-signed-certs
(x11)

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9dhnvms

Unhealthy

Startup probe failed: Get "https://10.132.0.44:8000/health": dial tcp 10.132.0.44:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-pvc-router-manage-59b9d263] is Ready
(x53)

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-9455cf8d5-6qghj

ProbeError

Startup probe error: Get "http://10.132.0.25:15021/healthz/ready": dial tcp 10.132.0.25:15021: connect: connection refused body:

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-b4dd96dbs59c7

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-b4dd96dbs59c7

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-7458955747-v8trl

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-b4dd96dbs59c7

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.133.0.50:9003" within 1s: context deadline exceeded

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-7458955747-v8trl

Unhealthy

Readiness probe failed: Get "https://10.132.0.42:8000/health": dial tcp 10.132.0.42:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schet9l7z

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9dhnvms

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schet9l7z

Killing

Stopping container main
(x4)

kserve-ci-e2e-test

gateway_labeler_controller

router-gateway-1

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway router-gateway-1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-testc9569cf4801efc0ed27b2f25ffaee875-llmisvc-mode

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Service kserve-ci-e2e-test/router-with-refs-test-kserve-workload-svc

kserve-ci-e2e-test

deployment-controller

router-with-refs-test-kserve

ScalingReplicaSet

Scaled up replica set router-with-refs-test-kserve-6b55745999 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Secret kserve-ci-e2e-test/router-with-refs-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Deployment kserve-ci-e2e-test/router-with-refs-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Role kserve-ci-e2e-test/router-with-refs-test-epp-role

kserve-ci-e2e-test

replicaset-controller

router-with-refs-test-kserve-6b55745999

SuccessfulCreate

Created pod: router-with-refs-test-kserve-6b55745999-xspsc

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/router-with-refs-test-epp-rb

kserve-ci-e2e-test

replicaset-controller

router-with-refs-test-kserve-router-scheduler-74b964bd8b

SuccessfulCreate

Created pod: router-with-refs-test-kserve-router-scheduler-74b964bd8b-zb6dt

kserve-ci-e2e-test

deployment-controller

router-with-refs-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set router-with-refs-test-kserve-router-scheduler-74b964bd8b from 0 to 1

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-6b55745999-xspsc

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-6b55745999-xspsc

Created

Created container: storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Service kserve-ci-e2e-test/router-with-refs-test-epp-service

kserve-ci-e2e-test

multus

router-with-refs-test-kserve-router-scheduler-74b964bd8b-zb6dt

AddedInterface

Add eth0 [10.133.0.53/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-74b964bd8b-zb6dt

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

multus

router-with-refs-test-kserve-6b55745999-xspsc

AddedInterface

Add eth0 [10.132.0.45/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-74b964bd8b-zb6dt

Created

Created container: storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Deployment kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-6b55745999-xspsc

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-74b964bd8b-zb6dt

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-74b964bd8b-zb6dt

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-74b964bd8b-zb6dt

Created

Created container: main

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-74b964bd8b-zb6dt

Started

Started container main
(x5)

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/router-with-refs-test-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-6b55745999-xspsc

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-6b55745999-xspsc

Started

Started container main

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-6b55745999-xspsc

Created

Created container: main
(x14)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-pvc-router-manage-e8706282

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-pvc-router-manage-e8706282-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-pvc-router-manage-e8706282-kserve-route" not found

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79497db4cc

SuccessfulCreate

Created pod: llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79zlqf4

kserve-ci-e2e-test

deployment-controller

llmisvc-model-pvc-router-manage-e8706282-kserve-prefill

ScalingReplicaSet

Scaled up replica set llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79497db4cc from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-testf5d060a5eb39a04e074b78907a1556a6-llmisvc-mode

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c

SuccessfulCreate

Created pod: llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-md987

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-prefill

kserve-ci-e2e-test

deployment-controller

llmisvc-model-pvc-router-manage-e8706282-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.Secret kserve-ci-e2e-test/llmisvb19f98874e050eec8ca94d49676113f0-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve

kserve-ci-e2e-test

kubelet

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schejwtjs

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schejwtjs

Started

Started container main

kserve-ci-e2e-test

multus

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schejwtjs

AddedInterface

Add eth0 [10.134.0.35/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheduler-67b4bb9646 from 0 to 1

kserve-ci-e2e-test

kubelet

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schejwtjs

Created

Created container: main

kserve-ci-e2e-test

replicaset-controller

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheduler-67b4bb9646

SuccessfulCreate

Created pod: llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schejwtjs

kserve-ci-e2e-test

attachdetach-controller

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schejwtjs

FailedAttachVolume

Multi-Attach error for volume "pvc-c58d8f9f-3948-479b-bcc9-16202a5f1201" Volume is already used by pod(s) llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-md987, llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79zlqf4

kserve-ci-e2e-test

multus

llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79zlqf4

AddedInterface

Add eth0 [10.132.0.47/23] from ovn-kubernetes

kserve-ci-e2e-test

multus

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-md987

AddedInterface

Add eth0 [10.132.0.46/23] from ovn-kubernetes

kserve-ci-e2e-test

attachdetach-controller

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-md987

SuccessfulAttachVolume

AttachVolume.Attach succeeded for volume "pvc-c58d8f9f-3948-479b-bcc9-16202a5f1201"

kserve-ci-e2e-test

attachdetach-controller

llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79zlqf4

SuccessfulAttachVolume

AttachVolume.Attach succeeded for volume "pvc-c58d8f9f-3948-479b-bcc9-16202a5f1201"

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-md987

Pulling

Pulling image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2"

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79zlqf4

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79zlqf4

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79zlqf4

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-md987

Started

Started container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-md987

Created

Created container: llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-md987

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" in 811ms (811ms including waiting). Image size: 44914394 bytes.

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-md987

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-md987

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-md987

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Updated

Updated v1.Secret kserve-ci-e2e-test/router-with-refs-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisvb19f98874e050eec8ca94d49676113f0-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-route
(x10)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-shadow-svc
(x13)

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-6b55745999-xspsc

Unhealthy

Startup probe failed: Get "https://10.132.0.45:8000/health": dial tcp 10.132.0.45:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

LLMInferenceServiceReady

LLMInferenceService [router-with-refs-test] is Ready

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-74b964bd8b-zb6dt

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-74b964bd8b-zb6dt

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-6b55745999-xspsc

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-test-epp-auth-rb
(x2)

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-6b55745999-xspsc

Unhealthy

Readiness probe failed: Get "https://10.132.0.45:8000/health": dial tcp 10.132.0.45:8000: connect: connection refused
(x11)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-50bc673d

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-50bc673d-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-50bc673d-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill

kserve-ci-e2e-test

replicaset-controller

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheduler-69454684c7

SuccessfulCreate

Created pod: llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schepcq9n

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-rb

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-76cbf745cc72pxw

Created

Created container: llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-76cbf745cc72pxw

Started

Started container llm-d-routing-sidecar

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-76cbf745cc72pxw

AddedInterface

Add eth0 [10.132.0.48/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-76cbf745cc

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-50bc673d-kserve-76cbf745cc72pxw

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.Secret kserve-ci-e2e-test/llmisv44d181485fad85e662eb092f3749502f-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-76cbf745cc72pxw

Pulled

Container image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

deployment-controller

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheduler-69454684c7 from 0 to 1

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-50bc673d-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-50bc673d-kserve-76cbf745cc from 0 to 1

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-649c8bb55d from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test00d7278d8a22c4e39146a6b0eb840f45-llmisvc-mode

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-649c8bb55d

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-64sg2zn

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-64sg2zn

AddedInterface

Add eth0 [10.132.0.49/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-64sg2zn

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-64sg2zn

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-64sg2zn

Started

Started container storage-initializer

kserve-ci-e2e-test

multus

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schepcq9n

AddedInterface

Add eth0 [10.133.0.54/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schepcq9n

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schepcq9n

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schepcq9n

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-76cbf745cc72pxw

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-76cbf745cc72pxw

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-76cbf745cc72pxw

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schepcq9n

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schepcq9n

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schepcq9n

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-76cbf745cc72pxw

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-76cbf745cc72pxw

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-64sg2zn

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-64sg2zn

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-64sg2zn

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-76cbf745cc72pxw

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv44d181485fad85e662eb092f3749502f-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-route
(x10)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-shadow-svc
(x17)

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79zlqf4

Unhealthy

Startup probe failed: Get "https://10.132.0.47:8000/health": dial tcp 10.132.0.47:8000: connect: connection refused
(x17)

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-md987

Unhealthy

Startup probe failed: Get "https://10.132.0.46:8001/health": dial tcp 10.132.0.46:8001: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-pvc-router-manage-e8706282] is Ready
(x12)

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-9455cf8d5-6qghj

Created

Created container: istio-proxy
(x17)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-76cbf745cc72pxw

Unhealthy

Startup probe failed: Get "https://10.132.0.48:8001/health": dial tcp 10.132.0.48:8001: connect: connection refused
(x17)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-64sg2zn

Unhealthy

Startup probe failed: Get "https://10.132.0.49:8000/health": dial tcp 10.132.0.49:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-fb-opt-125m-route-50bc673d] is Ready
(x11)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-pvc-router-manage-2577e794

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-pvc-router-manage-2577e794-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-pvc-router-manage-2577e794-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-epp-sa

kserve-ci-e2e-test

deployment-controller

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheduler-7dbcb75dbc from 0 to 1

kserve-ci-e2e-test

multus

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche9cdkj

AddedInterface

Add eth0 [10.134.0.36/23] from ovn-kubernetes

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-pvc-router-manage-2577e794-kserve-mn

CreatingRevision

Creating revision with key 686d468674 for a newly created LeaderWorkerSet

kserve-ci-e2e-test

kubelet

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche9cdkj

Created

Created container: main

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-pvc-router-manage-2577e794-kserve-mn

GroupsProgressing

Created leader statefulset llmisvc-model-pvc-router-manage-2577e794-kserve-mn
(x2)

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-pvc-router-manage-2577e794-kserve-mn

GroupsProgressing

Replicas are progressing, with 0 groups ready of total 1 groups

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-epp-rb

kserve-ci-e2e-test

statefulset-controller

llmisvc-model-pvc-router-manage-2577e794-kserve-mn

SuccessfulCreate

create Pod llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0 in StatefulSet llmisvc-model-pvc-router-manage-2577e794-kserve-mn successful

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-pvc-router-manage-2577e794-kserve-mn

GroupsProgressing

Created worker statefulset for leader pod llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

kserve-ci-e2e-test

kubelet

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche9cdkj

Started

Started container main

kserve-ci-e2e-test

replicaset-controller

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheduler-7dbcb75dbc

SuccessfulCreate

Created pod: llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche9cdkj

kserve-ci-e2e-test

multus

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1

AddedInterface

Add eth0 [10.132.0.51/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-scc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.LeaderWorkerSet kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.Secret kserve-ci-e2e-test/llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs

kserve-ci-e2e-test

multus

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

AddedInterface

Add eth0 [10.132.0.50/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

statefulset-controller

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

SuccessfulCreate

create Pod llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1 in StatefulSet llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0 successful

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn

kserve-ci-e2e-test

kubelet

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche9cdkj

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test37591b20e96e9663d45a730d03070f1e-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1

Created

Created container: main
(x183)

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-9455cf8d5-6qghj

BackOff

Back-off restarting failed container istio-proxy in pod openshift-ai-inference-openshift-default-9455cf8d5-6qghj_openshift-ingress(c37ebf5e-d20d-40c5-ba9f-9a9ded68ff76)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-route
(x8)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-shadow-svc
(x10)

kserve-ci-e2e-test

OpenDataHubModelController

custom-route-timeout-pd-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy custom-route-timeout-pd-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "custom-route-timeout-pd-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-rb

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Role kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-role

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Secret kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-self-signed-certs

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-pd-test-kserve-prefill-d75cf496c

SuccessfulCreate

Created pod: custom-route-timeout-pd-test-kserve-prefill-d75cf496c-lf5cr

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-pd-test-kserve-prefill

ScalingReplicaSet

Scaled up replica set custom-route-timeout-pd-test-kserve-prefill-d75cf496c from 0 to 1

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-pd-test-kserve

ScalingReplicaSet

Scaled up replica set custom-route-timeout-pd-test-kserve-db8bc57fd from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Service kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-workload-svc

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-pd-test-kserve-db8bc57fd

SuccessfulCreate

Created pod: custom-route-timeout-pd-test-kserve-db8bc57fd-9h9j7

kserve-ci-e2e-test

multus

custom-route-timeout-pd-test-kserve-router-scheduler-595d846bdl

AddedInterface

Add eth0 [10.133.0.56/23] from ovn-kubernetes

kserve-ci-e2e-test

multus

custom-route-timeout-pd-test-kserve-prefill-d75cf496c-lf5cr

AddedInterface

Add eth0 [10.132.0.52/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-pd-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-d75cf496c-lf5cr

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-595d846bdl

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-595d846bdl

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-595d846bdl

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-d75cf496c-lf5cr

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-d75cf496c-lf5cr

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

multus

custom-route-timeout-pd-test-kserve-db8bc57fd-9h9j7

AddedInterface

Add eth0 [10.133.0.55/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-db8bc57fd-9h9j7

Pulling

Pulling image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2"

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-pd-test-epp-sa

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-pd-test-kserve-router-scheduler-595d8dd66c

SuccessfulCreate

Created pod: custom-route-timeout-pd-test-kserve-router-scheduler-595d846bdl

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-pd-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set custom-route-timeout-pd-test-kserve-router-scheduler-595d8dd66c from 0 to 1

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-db8bc57fd-9h9j7

Created

Created container: llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-db8bc57fd-9h9j7

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" in 794ms (794ms including waiting). Image size: 44914394 bytes.

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-db8bc57fd-9h9j7

Started

Started container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-595d846bdl

Started

Started container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-595d846bdl

Created

Created container: main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-db8bc57fd-9h9j7

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-db8bc57fd-9h9j7

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-db8bc57fd-9h9j7

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-595d846bdl

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine
(x14)

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

Unhealthy

Startup probe failed: Get "https://10.132.0.50:8000/health": dial tcp 10.132.0.50:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-d75cf496c-lf5cr

Started

Started container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-d75cf496c-lf5cr

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-d75cf496c-lf5cr

Created

Created container: main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-db8bc57fd-9h9j7

Pulling

Pulling image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0"
(x2)

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-pvc-router-manage-2577e794-kserve-mn

AllGroupsReady

All replicas are ready, with 1 groups ready of total 1 groups

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-pvc-router-manage-2577e794] is Ready

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-db8bc57fd-9h9j7

Pulled

Successfully pulled image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" in 28.594s (28.594s including waiting). Image size: 3531177328 bytes.

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-db8bc57fd-9h9j7

Started

Started container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-db8bc57fd-9h9j7

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Updated

Updated v1.Secret kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-self-signed-certs
(x10)

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-shadow-svc
(x14)

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-9455cf8d5-6qghj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
(x13)

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-d75cf496c-lf5cr

Unhealthy

Startup probe failed: Get "https://10.132.0.52:8000/health": dial tcp 10.132.0.52:8000: connect: connection refused
(x12)

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-db8bc57fd-9h9j7

Unhealthy

Startup probe failed: Get "https://10.133.0.55:8001/health": dial tcp 10.133.0.55:8001: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

LLMInferenceServiceReady

LLMInferenceService [custom-route-timeout-pd-test] is Ready

kserve-ci-e2e-test

kubelet

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche9cdkj

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche9cdkj

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test37591b20e96e9663d45a730d03070f1e-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

Unhealthy

Readiness probe failed: Get "https://10.132.0.50:8000/health": dial tcp 10.132.0.50:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

conv-test-criticality

UpdateFailed

Failed to update status for LLMInferenceService "conv-test-criticality": Operation cannot be fulfilled on llminferenceservices.serving.kserve.io "conv-test-criticality": the object has been modified; please apply your changes to the latest version and try again

kserve-ci-e2e-test

LLMInferenceServiceController

conv-test-v1a1-to-v1a2

UpdateFailed

Failed to update status for LLMInferenceService "conv-test-v1a1-to-v1a2": llminferenceservices.serving.kserve.io "conv-test-v1a1-to-v1a2" not found

kserve-ci-e2e-test

LLMInferenceServiceController

conv-test-round-trip

UpdateFailed

Failed to update status for LLMInferenceService "conv-test-round-trip": Operation cannot be fulfilled on llminferenceservices.serving.kserve.io "conv-test-round-trip": the object has been modified; please apply your changes to the latest version and try again

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/stop-feature-test-epp-rb

kserve-ci-e2e-test

replicaset-controller

stop-feature-test-kserve-7fd76fc4cf

SuccessfulCreate

Created pod: stop-feature-test-kserve-7fd76fc4cf-2wz79

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7fd76fc4cf-2wz79

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Service kserve-ci-e2e-test/stop-feature-test-kserve-workload-svc

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7fd76fc4cf-2wz79

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7fd76fc4cf-2wz79

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

multus

stop-feature-test-kserve-7fd76fc4cf-2wz79

AddedInterface

Add eth0 [10.133.0.57/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Service kserve-ci-e2e-test/stop-feature-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Role kserve-ci-e2e-test/stop-feature-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-stop-feature-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/stop-feature-test-epp-sa

kserve-ci-e2e-test

deployment-controller

stop-feature-test-kserve

ScalingReplicaSet

Scaled up replica set stop-feature-test-kserve-7fd76fc4cf from 0 to 1

kserve-ci-e2e-test

deployment-controller

stop-feature-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set stop-feature-test-kserve-router-scheduler-664b88448d from 0 to 1

kserve-ci-e2e-test

replicaset-controller

stop-feature-test-kserve-router-scheduler-664b88448d

SuccessfulCreate

Created pod: stop-feature-test-kserve-router-scheduler-664b88448d-d7shc

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-664b88448d-d7shc

Pulling

Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743"

kserve-ci-e2e-test

multus

stop-feature-test-kserve-router-scheduler-664b88448d-d7shc

AddedInterface

Add eth0 [10.134.0.37/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-664b88448d-d7shc

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-664b88448d-d7shc

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-664b88448d-d7shc

Pulled

Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" in 3.723s (3.723s including waiting). Image size: 300570045 bytes.

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7fd76fc4cf-2wz79

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7fd76fc4cf-2wz79

Created

Created container: main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7fd76fc4cf-2wz79

Started

Started container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-664b88448d-d7shc

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-664b88448d-d7shc

Created

Created container: main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-664b88448d-d7shc

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Updated

Updated v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/stop-feature-test-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/stop-feature-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-pd-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-595d846bdl

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-595d846bdl

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-d75cf496c-lf5cr

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-db8bc57fd-9h9j7

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-db8bc57fd-9h9j7

Killing

Stopping container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-54c789bdc6-dhlsx

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8"

kserve-ci-e2e-test

deployment-controller

router-gateway-2-openshift-default

ScalingReplicaSet

Scaled up replica set router-gateway-2-openshift-default-54c789bdc6 from 0 to 1

kserve-ci-e2e-test

gateway_labeler_controller

router-gateway-2

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway router-gateway-2

kserve-ci-e2e-test

multus

router-gateway-2-openshift-default-54c789bdc6-dhlsx

AddedInterface

Add eth0 [10.134.0.38/23] from ovn-kubernetes

kserve-ci-e2e-test

service-controller

router-gateway-2-openshift-default

EnsuringLoadBalancer

Ensuring load balancer

kserve-ci-e2e-test

replicaset-controller

router-gateway-2-openshift-default-54c789bdc6

SuccessfulCreate

Created pod: router-gateway-2-openshift-default-54c789bdc6-dhlsx

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-54c789bdc6-dhlsx

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" in 2.991s (2.991s including waiting). Image size: 181251556 bytes.

kserve-ci-e2e-test

service-controller

router-gateway-2-openshift-default

EnsuredLoadBalancer

Ensured load balancer
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-54c789bdc6-dhlsx

Created

Created container: istio-proxy
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-54c789bdc6-dhlsx

Started

Started container istio-proxy

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-54c789bdc6-dhlsx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
(x6)

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-54c789bdc6-dhlsx

Unhealthy

Startup probe failed: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused

kserve-ci-e2e-test

deployment-controller

router-with-refs-pd-test-kserve

ScalingReplicaSet

Scaled up replica set router-with-refs-pd-test-kserve-765f8dd694 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/router-with-refs-pd-test-kserve-rb

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Role kserve-ci-e2e-test/router-with-refs-pd-test-kserve-role

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-pd-test-kserve

kserve-ci-e2e-test

replicaset-controller

router-with-refs-pd-test-kserve-765f8dd694

SuccessfulCreate

Created pod: router-with-refs-pd-test-kserve-765f8dd694-qtnc6

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Secret kserve-ci-e2e-test/router-with-refs-pd-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Deployment kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill

kserve-ci-e2e-test

replicaset-controller

router-with-refs-pd-test-kserve-prefill-5b6546bb4c

SuccessfulCreate

Created pod: router-with-refs-pd-test-kserve-prefill-5b6546bb4c-xrkp9

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5b6546bb4c-xrkp9

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5b6546bb4c-xrkp9

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

multus

router-with-refs-pd-test-kserve-prefill-5b6546bb4c-xrkp9

AddedInterface

Add eth0 [10.133.0.59/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-765f8dd694-qtnc6

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-765f8dd694-qtnc6

Created

Created container: storage-initializer

kserve-ci-e2e-test

multus

router-with-refs-pd-test-kserve-765f8dd694-qtnc6

AddedInterface

Add eth0 [10.133.0.58/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-765f8dd694-qtnc6

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-765f8dd694-qtnc6

Started

Started container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-765f8dd694-qtnc6

Created

Created container: llm-d-routing-sidecar

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-pd-test-epp-auth-rb

kserve-ci-e2e-test

deployment-controller

router-with-refs-pd-test-kserve-prefill

ScalingReplicaSet

Scaled up replica set router-with-refs-pd-test-kserve-prefill-5b6546bb4c from 0 to 1

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5b6546bb4c-xrkp9

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Deployment kserve-ci-e2e-test/router-with-refs-pd-test-kserve

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-765f8dd694-qtnc6

Pulled

Container image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Service kserve-ci-e2e-test/router-with-refs-pd-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-pd-test-epp-sa

kserve-ci-e2e-test

replicaset-controller

router-with-refs-pd-test-kserve-router-scheduler-56977fb9f5

SuccessfulCreate

Created pod: router-with-refs-pd-test-kserve-router-scheduler-56977fb9ftxcml

kserve-ci-e2e-test

multus

router-with-refs-pd-test-kserve-router-scheduler-56977fb9ftxcml

AddedInterface

Add eth0 [10.134.0.39/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-56977fb9ftxcml

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

deployment-controller

router-with-refs-pd-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set router-with-refs-pd-test-kserve-router-scheduler-56977fb9f5 from 0 to 1

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-56977fb9ftxcml

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-56977fb9ftxcml

Started

Started container storage-initializer
(x9)

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/router-with-refs-pd-test-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-56977fb9ftxcml

Started

Started container main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-56977fb9ftxcml

Created

Created container: main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-56977fb9ftxcml

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-765f8dd694-qtnc6

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-765f8dd694-qtnc6

Created

Created container: main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-765f8dd694-qtnc6

Started

Started container main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5b6546bb4c-xrkp9

Created

Created container: main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5b6546bb4c-xrkp9

Started

Started container main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5b6546bb4c-xrkp9

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine
(x12)

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7fd76fc4cf-2wz79

Unhealthy

Startup probe failed: Get "https://10.133.0.57:8000/health": dial tcp 10.133.0.57:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Updated

Updated v1.Secret kserve-ci-e2e-test/router-with-refs-pd-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-664b88448d-d7shc

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-664b88448d-d7shc

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.InferencePool kserve-ci-e2e-test/stop-feature-test-inference-pool

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.ServiceAccount kserve-ci-e2e-test/stop-feature-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Service kserve-ci-e2e-test/stop-feature-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Service kserve-ci-e2e-test/stop-feature-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7fd76fc4cf-2wz79

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

LLMInferenceServiceNotReady

LLMInferenceService [stop-feature-test] is no longer Ready because of: GatewaysReady, HTTPRoutesReady, InferencePoolReady, MainWorkloadReady, PrefillWorkerWorkloadReady, PrefillWorkloadReady, RouterReady, SchedulerWorkloadReady, WorkerWorkloadReady, WorkloadsReady
(x25)

kserve-ci-e2e-test

OpenDataHubModelController

stop-feature-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy stop-feature-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/stop-feature-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "stop-feature-test-kserve-route" not found

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7fd76fc4cf-2wz79

Unhealthy

Readiness probe failed: Get "https://10.133.0.57:8000/health": dial tcp 10.133.0.57:8000: connect: connection refused

kserve-ci-e2e-test

deployment-controller

stop-feature-test-kserve

ScalingReplicaSet

Scaled up replica set stop-feature-test-kserve-7fd76fc4cf from 0 to 1

kserve-ci-e2e-test

replicaset-controller

stop-feature-test-kserve-7fd76fc4cf

SuccessfulCreate

Created pod: stop-feature-test-kserve-7fd76fc4cf-rg7tw

kserve-ci-e2e-test

multus

stop-feature-test-kserve-7fd76fc4cf-rg7tw

AddedInterface

Add eth0 [10.133.0.60/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7fd76fc4cf-rg7tw

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-664b88448d-hvj9z

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7fd76fc4cf-rg7tw

Started

Started container storage-initializer

kserve-ci-e2e-test

deployment-controller

stop-feature-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set stop-feature-test-kserve-router-scheduler-664b88448d from 0 to 1

kserve-ci-e2e-test

multus

stop-feature-test-kserve-router-scheduler-664b88448d-hvj9z

AddedInterface

Add eth0 [10.134.0.40/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

stop-feature-test-kserve-router-scheduler-664b88448d

SuccessfulCreate

Created pod: stop-feature-test-kserve-router-scheduler-664b88448d-hvj9z

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-664b88448d-hvj9z

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7fd76fc4cf-rg7tw

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-664b88448d-hvj9z

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-664b88448d-hvj9z

Started

Started container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-664b88448d-hvj9z

Created

Created container: main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-664b88448d-hvj9z

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7fd76fc4cf-rg7tw

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7fd76fc4cf-rg7tw

Started

Started container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7fd76fc4cf-rg7tw

Created

Created container: main
(x17)

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-765f8dd694-qtnc6

Unhealthy

Startup probe failed: Get "https://10.133.0.58:8001/health": dial tcp 10.133.0.58:8001: connect: connection refused
(x18)

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5b6546bb4c-xrkp9

Unhealthy

Startup probe failed: Get "https://10.133.0.59:8000/health": dial tcp 10.133.0.59:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

LLMInferenceServiceReady

LLMInferenceService [router-with-refs-pd-test] is Ready
(x14)

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7fd76fc4cf-rg7tw

Unhealthy

Startup probe failed: Get "https://10.133.0.60:8000/health": dial tcp 10.133.0.60:8000: connect: connection refused
(x2)

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

LLMInferenceServiceReady

LLMInferenceService [stop-feature-test] is Ready

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7fd76fc4cf-rg7tw

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-664b88448d-hvj9z

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-664b88448d-hvj9z

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-7fd76fc4cf-rg7tw

Unhealthy

Readiness probe failed: Get "https://10.133.0.60:8000/health": dial tcp 10.133.0.60:8000: connect: connection refused
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

lora-single-adapter-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy lora-single-adapter-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/lora-single-adapter-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "lora-single-adapter-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

lora-single-adapter-test

Created

Created v1.Deployment kserve-ci-e2e-test/lora-single-adapter-test-kserve

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-77b8dc6f6b-qttbp

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

lora-single-adapter-test

Created

Created v1.Secret kserve-ci-e2e-test/lora-single-adapter-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

lora-single-adapter-test

Created

Created v1.Service kserve-ci-e2e-test/lora-single-adapter-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

lora-single-adapter-test

Created

Created v1.HTTPRoute kserve-ci-e2e-test/lora-single-adapter-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

lora-single-adapter-test

Created

Created v1.DestinationRule kserve-ci-e2e-test/lora-single-adapter-test-kserve-workload-svc

kserve-ci-e2e-test

replicaset-controller

lora-single-adapter-test-kserve-77b8dc6f6b

SuccessfulCreate

Created pod: lora-single-adapter-test-kserve-77b8dc6f6b-qttbp

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-77b8dc6f6b-qttbp

Created

Created container: storage-initializer

kserve-ci-e2e-test

deployment-controller

lora-single-adapter-test-kserve

ScalingReplicaSet

Scaled up replica set lora-single-adapter-test-kserve-77b8dc6f6b from 0 to 1

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-77b8dc6f6b-qttbp

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

multus

lora-single-adapter-test-kserve-77b8dc6f6b-qttbp

AddedInterface

Add eth0 [10.133.0.61/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-pd-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-56977fb9ftxcml

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-56977fb9ftxcml

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-765f8dd694-qtnc6

Killing

Stopping container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-5b6546bb4c-xrkp9

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-765f8dd694-qtnc6

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-77b8dc6f6b-qttbp

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-77b8dc6f6b-qttbp

Created

Created container: main

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-77b8dc6f6b-qttbp

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

lora-single-adapter-test

Updated

Updated v1.Secret kserve-ci-e2e-test/lora-single-adapter-test-kserve-self-signed-certs
(x15)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-3c960099

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-3c960099-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-3c960099-kserve-route" not found

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-3c960099-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-3c960099-kserve-74b7fb866d from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-3c960099

Created

Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-3c960099

Created

Created v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-3c960099

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-workload-svc

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-3c960099-kserve-74b7fb866d

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-3c960099-kserve-74b7fb866d7p7tf

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-3c960099

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-3c960099

Created

Created v1.Secret kserve-ci-e2e-test/llmisvd10b93c8eba12f87c3a350a5cae2ee0c-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-74b7fb866d7p7tf

Created

Created container: storage-initializer

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-3c960099-kserve-74b7fb866d7p7tf

AddedInterface

Add eth0 [10.133.0.62/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-74b7fb866d7p7tf

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-74b7fb866d7p7tf

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine
(x8)

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-77b8dc6f6b-qttbp

Unhealthy

Startup probe failed: Get "https://10.133.0.61:8000/health": dial tcp 10.133.0.61:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-74b7fb866d7p7tf

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-74b7fb866d7p7tf

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-74b7fb866d7p7tf

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-3c960099

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisvd10b93c8eba12f87c3a350a5cae2ee0c-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

lora-single-adapter-test

LLMInferenceServiceReady

LLMInferenceService [lora-single-adapter-test] is Ready

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-77b8dc6f6b-qttbp

Killing

Stopping container main
(x15)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-74b7fb866d7p7tf

Unhealthy

Startup probe failed: Get "https://10.133.0.62:8000/health": dial tcp 10.133.0.62:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-77b8dc6f6b-qttbp

Unhealthy

Readiness probe failed: Get "https://10.133.0.61:8000/health": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-3c960099

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-fb-opt-125m-route-3c960099] is Ready
(x10)

kserve-ci-e2e-test

OpenDataHubModelController

lora-multiple-adapters-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy lora-multiple-adapters-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "lora-multiple-adapters-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

lora-multiple-adapters-test

Created

Created v1.Service kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

lora-multiple-adapters-test

Created

Created v1.Deployment kserve-ci-e2e-test/lora-multiple-adapters-test-kserve

kserve-ci-e2e-test

deployment-controller

lora-multiple-adapters-test-kserve

ScalingReplicaSet

Scaled up replica set lora-multiple-adapters-test-kserve-556cfcc755 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

lora-multiple-adapters-test-kserve-556cfcc755

SuccessfulCreate

Created pod: lora-multiple-adapters-test-kserve-556cfcc755-lnsk6

kserve-ci-e2e-test

LLMInferenceServiceController

lora-multiple-adapters-test

Created

Created v1.Secret kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-556cfcc755-lnsk6

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-556cfcc755-lnsk6

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-556cfcc755-lnsk6

Created

Created container: storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

lora-multiple-adapters-test

Created

Created v1.DestinationRule kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-workload-svc

kserve-ci-e2e-test

multus

lora-multiple-adapters-test-kserve-556cfcc755-lnsk6

AddedInterface

Add eth0 [10.133.0.63/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

lora-multiple-adapters-test

Created

Created v1.HTTPRoute kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-route

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-556cfcc755-lnsk6

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-556cfcc755-lnsk6

Created

Created container: main

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-556cfcc755-lnsk6

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

lora-multiple-adapters-test

Updated

Updated v1.Secret kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-74b7fb866d7p7tf

Killing

Stopping container main
(x8)

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-556cfcc755-lnsk6

Unhealthy

Startup probe failed: Get "https://10.133.0.63:8000/health": dial tcp 10.133.0.63:8000: connect: connection refused
(x13)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-dc21cb14

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.Secret kserve-ci-e2e-test/llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-scc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn
(x2)

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

GroupsProgressing

Replicas are progressing, with 0 groups ready of total 1 groups

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schesfbnj

Started

Started container storage-initializer

kserve-ci-e2e-test

statefulset-controller

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

SuccessfulCreate

create Pod llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0 in StatefulSet llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn successful

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.LeaderWorkerSet kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test386bd5808c5c450f11fd8632e1f821fb-llmisvc-mode

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-epp-rb

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

AddedInterface

Add eth0 [10.132.0.53/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Created

Created container: storage-initializer

kserve-ci-e2e-test

statefulset-controller

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

SuccessfulCreate

create Pod llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1 in StatefulSet llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0 successful

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

GroupsProgressing

Created leader statefulset llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

AddedInterface

Add eth0 [10.133.0.64/23] from ovn-kubernetes

kserve-ci-e2e-test

multus

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schesfbnj

AddedInterface

Add eth0 [10.134.0.41/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-scheduler-6c66dbdc95

SuccessfulCreate

Created pod: llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schesfbnj

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schesfbnj

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schesfbnj

Created

Created container: storage-initializer

kserve-ci-e2e-test

deployment-controller

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-scheduler-6c66dbdc95 from 0 to 1

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

CreatingRevision

Creating revision with key bbd9c5c44 for a newly created LeaderWorkerSet

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Started

Started container storage-initializer

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

GroupsProgressing

Created worker statefulset for leader pod llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schesfbnj

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schesfbnj

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schesfbnj

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

lora-multiple-adapters-test

LLMInferenceServiceReady

LLMInferenceService [lora-multiple-adapters-test] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-route
(x8)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-556cfcc755-lnsk6

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-556cfcc755-lnsk6

Unhealthy

Readiness probe failed: Get "https://10.133.0.63:8000/health": dial tcp 10.133.0.63:8000: connect: connection refused

kserve-ci-e2e-test

replicaset-controller

tls-verification-test-kserve-6bbdc4587f

SuccessfulCreate

Created pod: tls-verification-test-kserve-6bbdc4587f-bc8c7

kserve-ci-e2e-test

deployment-controller

tls-verification-test-kserve

ScalingReplicaSet

Scaled up replica set tls-verification-test-kserve-6bbdc4587f from 0 to 1
(x10)

kserve-ci-e2e-test

OpenDataHubModelController

tls-verification-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy tls-verification-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/tls-verification-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "tls-verification-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.Secret kserve-ci-e2e-test/tls-verification-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.Deployment kserve-ci-e2e-test/tls-verification-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.Service kserve-ci-e2e-test/tls-verification-test-kserve-workload-svc

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-6bbdc4587f-bc8c7

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-6bbdc4587f-bc8c7

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.Service kserve-ci-e2e-test/tls-verification-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.Deployment kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler

kserve-ci-e2e-test

multus

tls-verification-test-kserve-6bbdc4587f-bc8c7

AddedInterface

Add eth0 [10.133.0.65/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/tls-verification-test-epp-rb

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-6bbdc4587f-bc8c7

Created

Created container: storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.Role kserve-ci-e2e-test/tls-verification-test-epp-role

kserve-ci-e2e-test

multus

tls-verification-test-kserve-router-scheduler-95f7d4548-8wsbk

AddedInterface

Add eth0 [10.134.0.42/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-tls-verification-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/tls-verification-test-epp-sa

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-95f7d4548-8wsbk

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-95f7d4548-8wsbk

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-95f7d4548-8wsbk

Started

Started container storage-initializer

kserve-ci-e2e-test

replicaset-controller

tls-verification-test-kserve-router-scheduler-95f7d4548

SuccessfulCreate

Created pod: tls-verification-test-kserve-router-scheduler-95f7d4548-8wsbk

kserve-ci-e2e-test

deployment-controller

tls-verification-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set tls-verification-test-kserve-router-scheduler-95f7d4548 from 0 to 1

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-95f7d4548-8wsbk

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-95f7d4548-8wsbk

Started

Started container main

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-95f7d4548-8wsbk

Created

Created container: main

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-6bbdc4587f-bc8c7

Created

Created container: main

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-6bbdc4587f-bc8c7

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-6bbdc4587f-bc8c7

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Updated

Updated v1.Secret kserve-ci-e2e-test/tls-verification-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/tls-verification-test-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/tls-verification-test-kserve-shadow-svc
(x11)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Unhealthy

Startup probe failed: Get "https://10.133.0.64:8000/health": dial tcp 10.133.0.64:8000: connect: connection refused
(x2)

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

AllGroupsReady

All replicas are ready, with 1 groups ready of total 1 groups
(x12)

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-6bbdc4587f-bc8c7

Unhealthy

Startup probe failed: Get "https://10.133.0.65:8000/health": dial tcp 10.133.0.65:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

LLMInferenceServiceReady

LLMInferenceService [tls-verification-test] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-tls-verification-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-95f7d4548-8wsbk

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-6bbdc4587f-bc8c7

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-95f7d4548-8wsbk

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-fb-opt-125m-route-dc21cb14] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/prestop-hook-test-epp-sa

kserve-ci-e2e-test

replicaset-controller

prestop-hook-test-kserve-7cc5b79f5

SuccessfulCreate

Created pod: prestop-hook-test-kserve-7cc5b79f5-rwtxg

kserve-ci-e2e-test

multus

prestop-hook-test-kserve-7cc5b79f5-rwtxg

AddedInterface

Add eth0 [10.133.0.66/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.Service kserve-ci-e2e-test/prestop-hook-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-prestop-hook-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.Role kserve-ci-e2e-test/prestop-hook-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/prestop-hook-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.Deployment kserve-ci-e2e-test/prestop-hook-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.Secret kserve-ci-e2e-test/prestop-hook-test-kserve-self-signed-certs

kserve-ci-e2e-test

deployment-controller

prestop-hook-test-kserve

ScalingReplicaSet

Scaled up replica set prestop-hook-test-kserve-7cc5b79f5 from 0 to 1

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-7cc5b79f5-rwtxg

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

deployment-controller

prestop-hook-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set prestop-hook-test-kserve-router-scheduler-577b99f89b from 0 to 1

kserve-ci-e2e-test

replicaset-controller

prestop-hook-test-kserve-router-scheduler-577b99f89b

SuccessfulCreate

Created pod: prestop-hook-test-kserve-router-scheduler-577b99f89b-wf8hm

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.Deployment kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.Service kserve-ci-e2e-test/prestop-hook-test-epp-service

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-577b99f89b-wf8hm

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

multus

prestop-hook-test-kserve-router-scheduler-577b99f89b-wf8hm

AddedInterface

Add eth0 [10.134.0.43/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-7cc5b79f5-rwtxg

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-7cc5b79f5-rwtxg

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-577b99f89b-wf8hm

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-577b99f89b-wf8hm

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-577b99f89b-wf8hm

Started

Started container main

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-577b99f89b-wf8hm

Created

Created container: main

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-577b99f89b-wf8hm

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-7cc5b79f5-rwtxg

Created

Created container: main

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-7cc5b79f5-rwtxg

Started

Started container main

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-7cc5b79f5-rwtxg

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/prestop-hook-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/prestop-hook-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test386bd5808c5c450f11fd8632e1f821fb-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schesfbnj

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schesfbnj

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Unhealthy

Readiness probe failed: Get "https://10.133.0.64:8000/health": dial tcp 10.133.0.64:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (llmisvc-model-fb-opt-125m-route-dc21cb-57cf7736-dockercfg-h7hzm); attempting to pull the image may not succeed.

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Killing

Stopping container main
(x12)

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-7cc5b79f5-rwtxg

Unhealthy

Startup probe failed: Get "https://10.133.0.66:8000/health": dial tcp 10.133.0.66:8000: connect: connection refused
(x6)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

FailedMount

MountVolume.SetUp failed for volume "tls-certs" : secret "llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs" not found

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

LLMInferenceServiceReady

LLMInferenceService [prestop-hook-test] is Ready

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-7cc5b79f5-rwtxg

Killing

Stopping container main

kserve-ci-e2e-test

replicaset-controller

prestop-hook-test-kserve-7cc5b79f5

SuccessfulCreate

Created pod: prestop-hook-test-kserve-7cc5b79f5-djkbl

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-7cc5b79f5-djkbl

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-7cc5b79f5-djkbl

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-7cc5b79f5-djkbl

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:33f82c067d520fe749f73749cf76b5935fe69bd9eb141504c6660698c7947743" already present on machine

kserve-ci-e2e-test

multus

prestop-hook-test-kserve-7cc5b79f5-djkbl

AddedInterface

Add eth0 [10.133.0.67/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-7cc5b79f5-djkbl

Started

Started container main

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-7cc5b79f5-djkbl

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-7cc5b79f5-djkbl

Created

Created container: main
(x2)

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Updated

Updated v1.Secret kserve-ci-e2e-test/prestop-hook-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

LLMInferenceServiceNotReady

LLMInferenceService [prestop-hook-test] is no longer Ready because of: MainWorkloadReady, WorkloadsReady
(x13)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-inline-config-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-inline-config-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-inline-config-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-inline-config-test-kserve-route" not found

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-7cc5b79f5-rwtxg

Unhealthy

Readiness probe failed: Get "https://10.133.0.66:8000/health": dial tcp 10.133.0.66:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-inline-config-test-kserve

kserve-ci-e2e-test

deployment-controller

scheduler-inline-config-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-inline-config-test-kserve-7c6c58cc96 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

scheduler-inline-config-test-kserve-7c6c58cc96

SuccessfulCreate

Created pod: scheduler-inline-config-test-kserve-7c6c58cc96-kbbbj

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-inline-config-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-7c6c58cc96-kbbbj

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-7c6c58cc96-kbbbj

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

replicaset-controller

scheduler-inline-config-test-kserve-router-scheduler-d9ddf6c4d

SuccessfulCreate

Created pod: scheduler-inline-config-test-kserve-router-scheduler-d9ddfhgsw6

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-d9ddfhgsw6

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-inline-config-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-inline-config-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler

kserve-ci-e2e-test

multus

scheduler-inline-config-test-kserve-router-scheduler-d9ddfhgsw6

AddedInterface

Add eth0 [10.134.0.45/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-d9ddfhgsw6

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-d9ddfhgsw6

Started

Started container main

kserve-ci-e2e-test

deployment-controller

scheduler-inline-config-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-inline-config-test-kserve-router-scheduler-d9ddf6c4d from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-inline-config-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-inline-config-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-inline-config-test-epp-role

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-7c6c58cc96-kbbbj

Created

Created container: main

kserve-ci-e2e-test

multus

scheduler-inline-config-test-kserve-7c6c58cc96-kbbbj

AddedInterface

Add eth0 [10.134.0.44/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-inline-config-test-epp-service
(x3)

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-7cc5b79f5-djkbl

Unhealthy

Startup probe failed: Get "https://10.133.0.67:8000/health": dial tcp 10.133.0.67:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-prestop-hook-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-577b99f89b-wf8hm

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-7cc5b79f5-djkbl

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-577b99f89b-wf8hm

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-577b99f89b-wf8hm

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-inline-config-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-inline-config-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-inline-config-test-kserve-route
(x13)

kserve-ci-e2e-test

OpenDataHubModelController

rolling-upgrade-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy rolling-upgrade-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/rolling-upgrade-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "rolling-upgrade-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/rolling-upgrade-test-epp-rb

kserve-ci-e2e-test

replicaset-controller

rolling-upgrade-test-kserve-54bc76f55b

SuccessfulCreate

Created pod: rolling-upgrade-test-kserve-54bc76f55b-xjr7c

kserve-ci-e2e-test

deployment-controller

rolling-upgrade-test-kserve

ScalingReplicaSet

Scaled up replica set rolling-upgrade-test-kserve-54bc76f55b from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.Secret kserve-ci-e2e-test/rolling-upgrade-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.Deployment kserve-ci-e2e-test/rolling-upgrade-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.Service kserve-ci-e2e-test/rolling-upgrade-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/rolling-upgrade-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-rolling-upgrade-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.Role kserve-ci-e2e-test/rolling-upgrade-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.Deployment kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.Service kserve-ci-e2e-test/rolling-upgrade-test-epp-service

kserve-ci-e2e-test

multus

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-6kfkp

AddedInterface

Add eth0 [10.133.0.68/23] from ovn-kubernetes

kserve-ci-e2e-test

multus

rolling-upgrade-test-kserve-54bc76f55b-xjr7c

AddedInterface

Add eth0 [10.134.0.46/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-54bc76f55b-xjr7c

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-54bc76f55b-xjr7c

Created

Created container: main

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-54bc76f55b-xjr7c

Started

Started container main

kserve-ci-e2e-test

deployment-controller

rolling-upgrade-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd from 0 to 1

kserve-ci-e2e-test

replicaset-controller

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd

SuccessfulCreate

Created pod: rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-6kfkp

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-6kfkp

Started

Started container main

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-6kfkp

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-6kfkp

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

LLMInferenceServiceReady

LLMInferenceService [scheduler-inline-config-test] is Ready
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/rolling-upgrade-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/rolling-upgrade-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

LLMInferenceServiceReady

LLMInferenceService [rolling-upgrade-test] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Updated

Updated v1.Deployment kserve-ci-e2e-test/rolling-upgrade-test-kserve

kserve-ci-e2e-test

deployment-controller

rolling-upgrade-test-kserve

ScalingReplicaSet

Scaled up replica set rolling-upgrade-test-kserve-859df57bcb from 0 to 1

kserve-ci-e2e-test

replicaset-controller

rolling-upgrade-test-kserve-859df57bcb

SuccessfulCreate

Created pod: rolling-upgrade-test-kserve-859df57bcb-m6kpq

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-859df57bcb-m6kpq

Started

Started container main

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-859df57bcb-m6kpq

Created

Created container: main

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-859df57bcb-m6kpq

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

multus

rolling-upgrade-test-kserve-859df57bcb-m6kpq

AddedInterface

Add eth0 [10.134.0.47/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-54bc76f55b-xjr7c

Killing

Stopping container main

kserve-ci-e2e-test

deployment-controller

rolling-upgrade-test-kserve

ScalingReplicaSet

Scaled down replica set rolling-upgrade-test-kserve-54bc76f55b from 1 to 0

kserve-ci-e2e-test

replicaset-controller

rolling-upgrade-test-kserve-54bc76f55b

SuccessfulDelete

Deleted pod: rolling-upgrade-test-kserve-54bc76f55b-xjr7c
(x2)

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Updated

Updated v1.Secret kserve-ci-e2e-test/rolling-upgrade-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-inline-config-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-7c6c58cc96-kbbbj

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-d9ddfhgsw6

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-d9ddfhgsw6

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-7c6c58cc96-kbbbj

Unhealthy

Readiness probe failed: Get "https://10.134.0.44:8000/health": dial tcp 10.134.0.44:8000: connect: connection refused
(x11)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-sa

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4flhjv

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.Secret kserve-ci-e2e-test/llmisv431d7e30f13554296364403a5b6365f5-kserve-self-signed-certs

kserve-ci-e2e-test

replicaset-controller

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scheduler-79866dc455

SuccessfulCreate

Created pod: llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schesnpfv

kserve-ci-e2e-test

deployment-controller

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scheduler-79866dc455 from 0 to 1

kserve-ci-e2e-test

multus

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schesnpfv

AddedInterface

Add eth0 [10.134.0.49/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schesnpfv

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schesnpfv

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test80f99418f0f3eb4c80f02e6144a6d4f2-llmisvc-mode

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-role

kserve-ci-e2e-test

deployment-controller

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4

SuccessfulCreate

Created pod: llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4flhjv

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scheduler

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4flhjv

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4flhjv

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-service

kserve-ci-e2e-test

multus

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4flhjv

AddedInterface

Add eth0 [10.134.0.48/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schesnpfv

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv431d7e30f13554296364403a5b6365f5-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-qwen2-5-0-5b-rout-4c1ba212] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test80f99418f0f3eb4c80f02e6144a6d4f2-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4flhjv

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schesnpfv

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schesnpfv

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4flhjv

Unhealthy

Readiness probe failed: Get "https://10.134.0.48:8000/health": dial tcp 10.134.0.48:8000: connect: connection refused
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-configmap-ref-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-configmap-ref-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-configmap-ref-test-kserve-route" not found

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-744bf59fbd-927xs

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-sa

kserve-ci-e2e-test

deployment-controller

scheduler-configmap-ref-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-configmap-ref-test-kserve-744bf59fbd from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-service

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-68f87gjvtx

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

multus

scheduler-configmap-ref-test-kserve-router-scheduler-68f87gjvtx

AddedInterface

Add eth0 [10.134.0.51/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

scheduler-configmap-ref-test-kserve-router-scheduler-68f876cc5d

SuccessfulCreate

Created pod: scheduler-configmap-ref-test-kserve-router-scheduler-68f87gjvtx

kserve-ci-e2e-test

replicaset-controller

scheduler-configmap-ref-test-kserve-744bf59fbd

SuccessfulCreate

Created pod: scheduler-configmap-ref-test-kserve-744bf59fbd-927xs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-workload-svc

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-744bf59fbd-927xs

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-744bf59fbd-927xs

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-rb

kserve-ci-e2e-test

multus

scheduler-configmap-ref-test-kserve-744bf59fbd-927xs

AddedInterface

Add eth0 [10.134.0.50/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-role

kserve-ci-e2e-test

deployment-controller

scheduler-configmap-ref-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-configmap-ref-test-kserve-router-scheduler-68f876cc5d from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-configmap-ref-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-68f87gjvtx

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-68f87gjvtx

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

LLMInferenceServiceReady

LLMInferenceService [scheduler-configmap-ref-test] is Ready

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-6kfkp

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-859df57bcb-m6kpq

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-rolling-upgrade-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-6kfkp

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-configmap-ref-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-68f87gjvtx

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-744bf59fbd-927xs

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-68f87gjvtx

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Error

Reconciliation failed: failed to reconcile workload: failed to reconcile self-signed certificates secret: failed to reconcile self-signed TLS certificate: failed to create v1.Secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs: context canceled
(x14)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-ha-replicas-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-ha-replicas-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-ha-replicas-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve

kserve-ci-e2e-test

replicaset-controller

scheduler-ha-replicas-test-kserve-5c5cfcf776

SuccessfulCreate

Created pod: scheduler-ha-replicas-test-kserve-5c5cfcf776-zt9gz

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs

kserve-ci-e2e-test

deployment-controller

scheduler-ha-replicas-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-ha-replicas-test-kserve-5c5cfcf776 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler

kserve-ci-e2e-test

replicaset-controller

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dbc4

SuccessfulCreate

Created pod: scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d7qf6j

kserve-ci-e2e-test

replicaset-controller

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dbc4

SuccessfulCreate

Created pod: scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dn2c6j

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dn2c6j

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-sa

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d7qf6j

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d7qf6j

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d7qf6j

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

deployment-controller

scheduler-ha-replicas-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dbc4 from 0 to 2

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dn2c6j

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-ha-replicas-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-5c5cfcf776-zt9gz

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-5c5cfcf776-zt9gz

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-5c5cfcf776-zt9gz

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

multus

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dn2c6j

AddedInterface

Add eth0 [10.134.0.53/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-role

kserve-ci-e2e-test

multus

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d7qf6j

AddedInterface

Add eth0 [10.133.0.70/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-rb

kserve-ci-e2e-test

multus

scheduler-ha-replicas-test-kserve-5c5cfcf776-zt9gz

AddedInterface

Add eth0 [10.134.0.52/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dn2c6j

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

LLMInferenceServiceReady

LLMInferenceService [scheduler-ha-replicas-test] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-ha-replicas-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dn2c6j

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dn2c6j

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-5c5cfcf776-zt9gz

Killing

Stopping container main
(x2)

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d7qf6j

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d7qf6j

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d7qf6j

FailedPreStopHook

PreStopHook failed
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-custom-template-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-custom-template-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-custom-template-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-custom-template-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-custom-template-test-epp-sa

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-5bc7bd6864-dm486

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-custom-template-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-custom-template-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-custom-template-test-epp-role

kserve-ci-e2e-test

replicaset-controller

scheduler-custom-template-test-kserve-router-scheduler-567469f665

SuccessfulCreate

Created pod: scheduler-custom-template-test-kserve-router-scheduler-567pxl6w

kserve-ci-e2e-test

replicaset-controller

scheduler-custom-template-test-kserve-5bc7bd6864

SuccessfulCreate

Created pod: scheduler-custom-template-test-kserve-5bc7bd6864-dm486

kserve-ci-e2e-test

deployment-controller

scheduler-custom-template-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-custom-template-test-kserve-5bc7bd6864 from 0 to 1

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-5bc7bd6864-dm486

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-5bc7bd6864-dm486

Created

Created container: main

kserve-ci-e2e-test

deployment-controller

scheduler-custom-template-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-custom-template-test-kserve-router-scheduler-567469f665 from 0 to 1

kserve-ci-e2e-test

multus

scheduler-custom-template-test-kserve-5bc7bd6864-dm486

AddedInterface

Add eth0 [10.134.0.54/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-custom-template-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-custom-template-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-custom-template-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-custom-template-test-epp-auth-rb

kserve-ci-e2e-test

multus

scheduler-custom-template-test-kserve-router-scheduler-567pxl6w

AddedInterface

Add eth0 [10.134.0.55/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-router-scheduler-567pxl6w

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-router-scheduler-567pxl6w

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-router-scheduler-567pxl6w

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-custom-template-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-custom-template-test-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-custom-template-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

LLMInferenceServiceReady

LLMInferenceService [scheduler-custom-template-test] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-custom-template-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-router-scheduler-567pxl6w

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-router-scheduler-567pxl6w

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-5bc7bd6864-dm486

Killing

Stopping container main
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-v06-pd-migration-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-v06-pd-migration-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-v06-pd-migration-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-role

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-workload-svc

kserve-ci-e2e-test

deployment-controller

scheduler-v06-pd-migration-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-v06-pd-migration-test-kserve-5c87756974 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-v06-pd-migration-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-pd-migration-test-epp-sa

kserve-ci-e2e-test

replicaset-controller

scheduler-v06-pd-migration-test-kserve-5c87756974

SuccessfulCreate

Created pod: scheduler-v06-pd-migration-test-kserve-5c87756974-pvgch

kserve-ci-e2e-test

deployment-controller

scheduler-v06-pd-migration-test-kserve-prefill

ScalingReplicaSet

Scaled up replica set scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847

SuccessfulCreate

Created pod: scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-dq68c

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-router-scheduler-59kgmqp

Started

Started container main

kserve-ci-e2e-test

deployment-controller

scheduler-v06-pd-migration-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-v06-pd-migration-test-kserve-router-scheduler-59dc679cd from 0 to 1

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-dq68c

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-dq68c

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

multus

scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-dq68c

AddedInterface

Add eth0 [10.134.0.57/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

scheduler-v06-pd-migration-test-kserve-router-scheduler-59dc679cd

SuccessfulCreate

Created pod: scheduler-v06-pd-migration-test-kserve-router-scheduler-59kgmqp

kserve-ci-e2e-test

multus

scheduler-v06-pd-migration-test-kserve-router-scheduler-59kgmqp

AddedInterface

Add eth0 [10.134.0.58/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-router-scheduler-59kgmqp

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-router-scheduler-59kgmqp

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-dq68c

Started

Started container main

kserve-ci-e2e-test

multus

scheduler-v06-pd-migration-test-kserve-5c87756974-pvgch

AddedInterface

Add eth0 [10.134.0.56/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-pvgch

Pulling

Pulling image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2"

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-pvgch

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-pvgch

Started

Started container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-pvgch

Created

Created container: llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-pvgch

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" in 767ms (767ms including waiting). Image size: 44914394 bytes.

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-pvgch

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-pvgch

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-self-signed-certs
(x10)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

LLMInferenceServiceReady

LLMInferenceService [scheduler-v06-pd-migration-test] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-v06-pd-migration-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-router-scheduler-59kgmqp

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-router-scheduler-59kgmqp

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-dq68c

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-pvgch

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-pvgch

Killing

Stopping container llm-d-routing-sidecar
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-v06-threshold-migration-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-v06-threshold-migration-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-v06-threshold-migration-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test0a07e82be2036793543b2d3db2ea0785-scheduler-v0

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-role

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-self-signed-certs

kserve-ci-e2e-test

replicaset-controller

scheduler-v06-threshold-migration-test-kserve-5994569d5b

SuccessfulCreate

Created pod: scheduler-v06-threshold-migration-test-kserve-5994569d5b-sqwkm

kserve-ci-e2e-test

replicaset-controller

scheduler-v06-threshold-migration-test-kserve-prefill-5fc599dc49

SuccessfulCreate

Created pod: scheduler-v06-threshold-migration-test-kserve-prefill-5fc5st7gp

kserve-ci-e2e-test

deployment-controller

scheduler-v06-threshold-migration-test-kserve-prefill

ScalingReplicaSet

Scaled up replica set scheduler-v06-threshold-migration-test-kserve-prefill-5fc599dc49 from 0 to 1

kserve-ci-e2e-test

deployment-controller

scheduler-v06-threshold-migration-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-v06-threshold-migration-test-kserve-5994569d5b from 0 to 1

kserve-ci-e2e-test

deployment-controller

scheduler-v06-threshold-migration-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-v06-threshold-migration-test-kserve-router-scheduler-dbc65f645 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

scheduler-v06-threshold-migration-test-kserve-router-scheduler-dbc65f645

SuccessfulCreate

Created pod: scheduler-v06-threshold-migration-test-kserve-router-schedctr8m

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-sqwkm

Created

Created container: llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-prefill-5fc5st7gp

Created

Created container: main

kserve-ci-e2e-test

multus

scheduler-v06-threshold-migration-test-kserve-router-schedctr8m

AddedInterface

Add eth0 [10.134.0.61/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-router-schedctr8m

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-router-schedctr8m

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-router-schedctr8m

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-sqwkm

Pulled

Container image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-sqwkm

Started

Started container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-prefill-5fc5st7gp

Started

Started container main

kserve-ci-e2e-test

multus

scheduler-v06-threshold-migration-test-kserve-5994569d5b-sqwkm

AddedInterface

Add eth0 [10.134.0.59/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-prefill-5fc5st7gp

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

multus

scheduler-v06-threshold-migration-test-kserve-prefill-5fc5st7gp

AddedInterface

Add eth0 [10.134.0.60/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-sqwkm

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-sqwkm

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-sqwkm

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-self-signed-certs
(x10)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

LLMInferenceServiceReady

LLMInferenceService [scheduler-v06-threshold-migration-test] is Ready

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-router-schedctr8m

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test0a07e82be2036793543b2d3db2ea0785-scheduler-v0

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-sqwkm

Killing

Stopping container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-sqwkm

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-prefill-5fc5st7gp

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-router-schedctr8m

Killing

Stopping container main