Time Namespace Component RelatedObject Reason Message

kserve-ci-e2e-test

router-with-refs-pd-test-kserve-prefill-9d68b57d4-trssd

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-9d68b57d4-trssd to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

router-with-refs-pd-test-kserve-664bd66db5-cjjg6

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-664bd66db5-cjjg6 to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-fwwrx

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-fwwrx to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

auth-disabled-test-kserve-ccd54d6d4-cgwk9

Scheduled

Successfully assigned kserve-ci-e2e-test/auth-disabled-test-kserve-ccd54d6d4-cgwk9 to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dbch5s

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dbch5s to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

scheduler-v06-pd-migration-test-kserve-5c87756974-hnm4k

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-hnm4k to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-6ppgx

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-6ppgx to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

scheduler-v06-pd-migration-test-kserve-router-scheduler-59pfml2

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-59pfml2 to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

scheduler-v06-threshold-migration-test-kserve-5994569d5b-t8xj9

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-t8xj9 to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ncl6m

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ncl6m to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879ntk8x

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879ntk8x to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

stop-feature-test-kserve-5f875cbf68-7df7h

Scheduled

Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-5f875cbf68-7df7h to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

auth-disabled-test-kserve-router-scheduler-58f5ddbc8f-qgm5g

Scheduled

Successfully assigned kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-58f5ddbc8f-qgm5g to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

stop-feature-test-kserve-5f875cbf68-x2lqh

Scheduled

Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-5f875cbf68-x2lqh to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

stop-feature-test-kserve-router-scheduler-76cf8bfb8f-75mhj

Scheduled

Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-76cf8bfb8f-75mhj to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d9qw9w

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d9qw9w to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

stop-feature-test-kserve-router-scheduler-76cf8bfb8f-qgsf9

Scheduled

Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-76cf8bfb8f-qgsf9 to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

tls-verification-test-kserve-759794d6b4-9m772

Scheduled

Successfully assigned kserve-ci-e2e-test/tls-verification-test-kserve-759794d6b4-9m772 to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

tls-verification-test-kserve-router-scheduler-5db688dfd5-ngfb8

Scheduled

Successfully assigned kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5db688dfd5-ngfb8 to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-3c960099-kserve-5c94bf7bb75w9xs

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-5c94bf7bb75w9xs to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

prestop-hook-test-kserve-65cf7fd568-dnqbs

Scheduled

Successfully assigned kserve-ci-e2e-test/prestop-hook-test-kserve-65cf7fd568-dnqbs to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-84599bcd6b4vvr6

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-84599bcd6b4vvr6 to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

scheduler-ha-replicas-test-kserve-5c5cfcf776-9djm5

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-9djm5 to ip-10-0-139-134.ec2.internal

kuadrant-system

authorino-7c77957575-cfthr

Scheduled

Successfully assigned kuadrant-system/authorino-7c77957575-cfthr to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-6f654d9

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-6f654d9 to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

precise-prefix-cache-test-kserve-router-scheduler-d84595fbj5ftw

Scheduled

Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-d84595fbj5ftw to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

precise-prefix-cache-test-kserve-6ffc955574-v494f

Scheduled

Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6ffc955574-v494f to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5db6d9f47djrd27

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5db6d9f47djrd27 to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

precise-prefix-cache-test-kserve-6ffc955574-p26g2

Scheduled

Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6ffc955574-p26g2 to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b4jkrn

Scheduled

Successfully assigned kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b4jkrn to ip-10-0-128-215.ec2.internal

kuadrant-system

authorino-8df697c9f-brb5n

Scheduled

Successfully assigned kuadrant-system/authorino-8df697c9f-brb5n to ip-10-0-139-134.ec2.internal

openshift-ingress

istiod-openshift-gateway-94bb8fbfd-zprl6

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-94bb8fbfd-zprl6 to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

router-gateway-1-openshift-default-54c47cdfc7-lfprg

Scheduled

Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-lfprg to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

lora-single-adapter-test-kserve-5849bd8489-gtjrb

Scheduled

Successfully assigned kserve-ci-e2e-test/lora-single-adapter-test-kserve-5849bd8489-gtjrb to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-pd-test-kserve-prefill-f7f58c4f6-nsctq

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-f7f58c4f6-nsctq to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

router-gateway-1-openshift-default-54c47cdfc7-r2bjb

Scheduled

Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-r2bjb to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

router-gateway-1-openshift-default-54c47cdfc7-kgtbh

Scheduled

Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kgtbh to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-pd-test-kserve-router-scheduler-5f5fcfmn6z

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5f5fcfmn6z to ip-10-0-131-203.ec2.internal

openshift-ingress

router-default-586b57644b-q24lm

Scheduled

Successfully assigned openshift-ingress/router-default-586b57644b-q24lm to ip-10-0-139-134.ec2.internal

kuadrant-system

authorino-b6c89bb85-knflr

Scheduled

Successfully assigned kuadrant-system/authorino-b6c89bb85-knflr to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

auth-enabled-test-kserve-7dfddf4646-c84nz

Scheduled

Successfully assigned kserve-ci-e2e-test/auth-enabled-test-kserve-7dfddf4646-c84nz to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schelnjhn

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schelnjhn to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1 to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0 to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

router-gateway-1-openshift-default-799f46c59b-fqc69

Scheduled

Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-fqc69 to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-pd-test-kserve-6d9d59446f-spptn

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-6d9d59446f-spptn to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0 to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9dmww7f

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9dmww7f to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

router-gateway-1-openshift-default-799f46c59b-kkqwb

Scheduled

Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kkqwb to ip-10-0-131-203.ec2.internal

openshift-ingress

openshift-ai-inference-openshift-default-655f7c58b4-8g97n

Scheduled

Successfully assigned openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-8g97n to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc5cktt6

Scheduled

Successfully assigned kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc5cktt6 to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

prestop-hook-test-kserve-65cf7fd568-kgw6w

Scheduled

Successfully assigned kserve-ci-e2e-test/prestop-hook-test-kserve-65cf7fd568-kgw6w to ip-10-0-131-203.ec2.internal

kuadrant-system

dns-operator-controller-manager-65b49595d7-hphfv

Scheduled

Successfully assigned kuadrant-system/dns-operator-controller-manager-65b49595d7-hphfv to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

auth-enabled-test-kserve-router-scheduler-7b8b8fbfcb-h298k

Scheduled

Successfully assigned kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-7b8b8fbfcb-h298k to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-mhsdz

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-mhsdz to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79j8gwt

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79j8gwt to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-dfnlm

Scheduled

Successfully assigned kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-dfnlm to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-794bb95d4d-s6j8f

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-794bb95d4d-s6j8f to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scher2z97

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scher2z97 to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4hjlzp

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4hjlzp to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fb2h9x

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fb2h9x to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddcrqxt

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddcrqxt to ip-10-0-139-134.ec2.internal

kuadrant-system

limitador-operator-controller-manager-6f9f468797-mw8p8

Scheduled

Successfully assigned kuadrant-system/limitador-operator-controller-manager-6f9f468797-mw8p8 to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

e2e-pvc-model-download-w2mvs

Scheduled

Successfully assigned kserve-ci-e2e-test/e2e-pvc-model-download-w2mvs to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

scheduler-inline-config-test-kserve-7c6c58cc96-t4d7s

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-t4d7s to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

scheduler-inline-config-test-kserve-router-scheduler-d9ddfrbp26

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrbp26 to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

scheduler-v06-threshold-migration-test-kserve-router-schedz7ptr

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-schedz7ptr to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-79c7c55747-lfmdf

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-79c7c55747-lfmdf to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4ctbbm

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4ctbbm to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schecjg9g

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schecjg9g to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-schelqsd2

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-schelqsd2 to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schefs4wn

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schefs4wn to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche6fld4

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche6fld4 to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schenm4bp

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schenm4bp to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schel7mgq

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schel7mgq to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemgcff

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemgcff to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scherchj5

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scherchj5 to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche7bbzq

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche7bbzq to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehlrzb

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehlrzb to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

prestop-hook-test-kserve-router-scheduler-86bc684b85-qqhf2

Scheduled

Successfully assigned kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-86bc684b85-qqhf2 to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

scheduler-custom-template-test-kserve-router-scheduler-567bmx62

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567bmx62 to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

auth-invalid-token-test-kserve-58b5cf756b-27d2s

Scheduled

Successfully assigned kserve-ci-e2e-test/auth-invalid-token-test-kserve-58b5cf756b-27d2s to ip-10-0-128-215.ec2.internal

kuadrant-system

limitador-limitador-ff68ff5cb-4ph8z

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-ff68ff5cb-4ph8z to ip-10-0-131-203.ec2.internal

kuadrant-system

limitador-limitador-69574b596d-hnvp6

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-69574b596d-hnvp6 to ip-10-0-128-215.ec2.internal

kuadrant-system

authorino-operator-6d85f6564-xdn96

Scheduled

Successfully assigned kuadrant-system/authorino-operator-6d85f6564-xdn96 to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-test-kserve-router-scheduler-685c7787ffkr6

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-685c7787ffkr6 to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

rolling-upgrade-test-kserve-54bc76f55b-xjzqz

Scheduled

Successfully assigned kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-xjzqz to ip-10-0-139-134.ec2.internal

kuadrant-system

kuadrant-operator-controller-manager-bb7ddb7d7-cf52x

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-cf52x to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

custom-route-timeout-test-kserve-965ccd984-nft7b

Scheduled

Successfully assigned kserve-ci-e2e-test/custom-route-timeout-test-kserve-965ccd984-nft7b to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

scheduler-custom-template-test-kserve-5bc7bd6864-28w56

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-28w56 to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

scheduler-configmap-ref-test-kserve-router-scheduler-68f87bprxs

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87bprxs to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

scheduler-configmap-ref-test-kserve-744bf59fbd-chn2n

Scheduled

Successfully assigned kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-chn2n to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

router-with-refs-test-kserve-router-scheduler-7f865578d7-hrblf

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7f865578d7-hrblf to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

router-with-refs-test-kserve-555479964d-4kccs

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-test-kserve-555479964d-4kccs to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

auth-invalid-token-test-kserve-router-scheduler-6c486c59585thrg

Scheduled

Successfully assigned kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-6c486c59585thrg to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

router-with-refs-pd-test-kserve-router-scheduler-5c75cd7b8gcvgz

Scheduled

Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5c75cd7b8gcvgz to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1 to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4jt7jj

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4jt7jj to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

router-gateway-2-openshift-default-684dc9b4b9-c9txx

Scheduled

Successfully assigned kserve-ci-e2e-test/router-gateway-2-openshift-default-684dc9b4b9-c9txx to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

router-gateway-2-openshift-default-54c789bdc6-chk28

Scheduled

Successfully assigned kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-chk28 to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schex5jnk

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schex5jnk to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

rolling-upgrade-test-kserve-859df57bcb-7z6cd

Scheduled

Successfully assigned kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-7z6cd to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schek22wf

Scheduled

Successfully assigned kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schek22wf to ip-10-0-139-134.ec2.internal

kserve-ci-e2e-test

lora-multiple-adapters-test-kserve-66df458ff4-5ntmf

Scheduled

Successfully assigned kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-66df458ff4-5ntmf to ip-10-0-131-203.ec2.internal

kserve-ci-e2e-test

router-gateway-1-openshift-default-799f46c59b-rbgv4

Scheduled

Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-rbgv4 to ip-10-0-131-203.ec2.internal

kuadrant-system

kuadrant-console-plugin-85fd4b8b9d-4tc92

Scheduled

Successfully assigned kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-4tc92 to ip-10-0-128-215.ec2.internal

kserve-ci-e2e-test

router-gateway-1-openshift-default-799f46c59b-s6nj7

Scheduled

Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-s6nj7 to ip-10-0-139-134.ec2.internal

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-586b57644b from 0 to 1

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer
(x14)

openshift-ingress

replicaset-controller

router-default-586b57644b

FailedCreate

Error creating: pods "router-default-586b57644b-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x16)

openshift-ingress

replicaset-controller

router-default-586b57644b

FailedCreate

Error creating: pods "router-default-586b57644b-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts

openshift-ingress

replicaset-controller

router-default-586b57644b

SuccessfulCreate

Created pod: router-default-586b57644b-q24lm

openshift-ingress

kubelet

router-default-586b57644b-q24lm

FailedMount

MountVolume.SetUp failed for volume "stats-auth" : failed to sync secret cache: timed out waiting for the condition
(x5)

openshift-ingress

kubelet

router-default-586b57644b-q24lm

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x5)

openshift-ingress

kubelet

router-default-586b57644b-q24lm

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt

openshift-ingress

kubelet

router-default-586b57644b-q24lm

Started

Started container router

openshift-ingress

kubelet

router-default-586b57644b-q24lm

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0595b1d6de8e08ddb61ce581075357331193db6206d341c15cf213514ef0f693" already present on machine

openshift-ingress

kubelet

router-default-586b57644b-q24lm

Created

Created container: router

openshift-ingress

multus

router-default-586b57644b-q24lm

AddedInterface

Add eth0 [10.133.0.11/23] from ovn-kubernetes

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-94bb8fbfd from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-94bb8fbfd

SuccessfulCreate

Created pod: istiod-openshift-gateway-94bb8fbfd-zprl6

openshift-ingress

kubelet

istiod-openshift-gateway-94bb8fbfd-zprl6

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0850242436e88f7d82f0f2126de064c7e0f09844f31d8ff0f53dc8d3908075d9"

openshift-ingress

multus

istiod-openshift-gateway-94bb8fbfd-zprl6

AddedInterface

Add eth0 [10.132.0.19/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-94bb8fbfd-zprl6

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0850242436e88f7d82f0f2126de064c7e0f09844f31d8ff0f53dc8d3908075d9" in 2.536s (2.536s including waiting). Image size: 181034515 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-94bb8fbfd-zprl6

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-94bb8fbfd-zprl6

Created

Created container: discovery

openshift-ingress

service-controller

openshift-ai-inference-openshift-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

deployment-controller

openshift-ai-inference-openshift-default

ScalingReplicaSet

Scaled up replica set openshift-ai-inference-openshift-default-655f7c58b4 from 0 to 1

openshift-ingress

replicaset-controller

openshift-ai-inference-openshift-default-655f7c58b4

SuccessfulCreate

Created pod: openshift-ai-inference-openshift-default-655f7c58b4-8g97n

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-655f7c58b4-8g97n

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8"

openshift-ingress

multus

openshift-ai-inference-openshift-default-655f7c58b4-8g97n

AddedInterface

Add eth0 [10.134.0.33/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

openshift-ai-inference

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway openshift-ai-inference

openshift-ingress

service-controller

openshift-ai-inference-openshift-default

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-655f7c58b4-8g97n

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" in 4.179s (4.179s including waiting). Image size: 181251556 bytes.

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.0

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.0

RequirementsNotMet

one or more requirements couldn't be found
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.0

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.0

InstallWaiting

installing: waiting for deployment limitador-operator-controller-manager to become ready: deployment "limitador-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

multus

limitador-operator-controller-manager-6f9f468797-mw8p8

AddedInterface

Add eth0 [10.132.0.24/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-operator-controller-manager-6f9f468797-mw8p8

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:183bad2a6f5872aea38f54ad507f7e005d6facb02b18769e0407c82799f96a3e"

kuadrant-system

replicaset-controller

limitador-operator-controller-manager-6f9f468797

SuccessfulCreate

Created pod: limitador-operator-controller-manager-6f9f468797-mw8p8

kuadrant-system

deployment-controller

limitador-operator-controller-manager

ScalingReplicaSet

Scaled up replica set limitador-operator-controller-manager-6f9f468797 from 0 to 1

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.1

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.1

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.0

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

kubelet

limitador-operator-controller-manager-6f9f468797-mw8p8

Started

Started container manager

kuadrant-system

kubelet

limitador-operator-controller-manager-6f9f468797-mw8p8

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:183bad2a6f5872aea38f54ad507f7e005d6facb02b18769e0407c82799f96a3e" in 2.714s (2.714s including waiting). Image size: 178538972 bytes.

kuadrant-system

kubelet

limitador-operator-controller-manager-6f9f468797-mw8p8

Created

Created container: manager

kuadrant-system

limitador-operator-controller-manager-6f9f468797-mw8p8_a2be9359-31a8-465e-98f8-2ef5c855f163

3745a16e.kuadrant.io

LeaderElection

limitador-operator-controller-manager-6f9f468797-mw8p8_a2be9359-31a8-465e-98f8-2ef5c855f163 became leader

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.1

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.1

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.1

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.1

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.1

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-bb7ddb7d7 from 0 to 1

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-bb7ddb7d7

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-bb7ddb7d7-cf52x

kuadrant-system

multus

kuadrant-operator-controller-manager-bb7ddb7d7-cf52x

AddedInterface

Add eth0 [10.132.0.25/23] from ovn-kubernetes

kuadrant-system

kubelet

kuadrant-operator-controller-manager-bb7ddb7d7-cf52x

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:0b77e4a874b620095a0300877aea53e78559a601ce62b194537ed21c2743bbf3"

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.0

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

replicaset-controller

authorino-operator-6d85f6564

SuccessfulCreate

Created pod: authorino-operator-6d85f6564-xdn96

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.0

RequirementsUnknown

requirements not yet checked

kuadrant-system

multus

authorino-operator-6d85f6564-xdn96

AddedInterface

Add eth0 [10.133.0.28/23] from ovn-kubernetes

kuadrant-system

deployment-controller

authorino-operator

ScalingReplicaSet

Scaled up replica set authorino-operator-6d85f6564 from 0 to 1

kuadrant-system

kubelet

authorino-operator-6d85f6564-xdn96

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:b2cc2c0f0729dad98f970830816e300c0b961ba42491f362295512d886384687"

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.1

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.1

InstallWaiting

installing: waiting for deployment authorino-operator to become ready: deployment "authorino-operator" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.1

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.0

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

kubelet

kuadrant-operator-controller-manager-bb7ddb7d7-cf52x

Created

Created container: manager

kuadrant-system

replicaset-controller

dns-operator-controller-manager-65b49595d7

SuccessfulCreate

Created pod: dns-operator-controller-manager-65b49595d7-hphfv

kuadrant-system

deployment-controller

dns-operator-controller-manager

ScalingReplicaSet

Scaled up replica set dns-operator-controller-manager-65b49595d7 from 0 to 1

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.0

InstallWaiting

installing: waiting for deployment dns-operator-controller-manager to become ready: deployment "dns-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

kubelet

kuadrant-operator-controller-manager-bb7ddb7d7-cf52x

Started

Started container manager

kuadrant-system

kubelet

kuadrant-operator-controller-manager-bb7ddb7d7-cf52x

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:0b77e4a874b620095a0300877aea53e78559a601ce62b194537ed21c2743bbf3" in 4.718s (4.718s including waiting). Image size: 425661757 bytes.

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.0

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

kubelet

authorino-operator-6d85f6564-xdn96

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:b2cc2c0f0729dad98f970830816e300c0b961ba42491f362295512d886384687" in 2.594s (2.594s including waiting). Image size: 168639631 bytes.

kuadrant-system

authorino-operator-6d85f6564-xdn96_64e2d26b-2e90-4e06-b2ee-726971ea0321

aac3a15d.authorino.kuadrant.io

LeaderElection

authorino-operator-6d85f6564-xdn96_64e2d26b-2e90-4e06-b2ee-726971ea0321 became leader

kuadrant-system

kuadrant-operator-controller-manager-bb7ddb7d7-cf52x_c0802d73-e190-44c0-8f6a-68c2edeb5360

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-bb7ddb7d7-cf52x_c0802d73-e190-44c0-8f6a-68c2edeb5360 became leader

kuadrant-system

kubelet

dns-operator-controller-manager-65b49595d7-hphfv

Pulling

Pulling image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:ba46f54446ecd3ed1a3d877dcbb7359ff8f2d918827f3411a5cb0781915a65d5"

kuadrant-system

kubelet

authorino-operator-6d85f6564-xdn96

Started

Started container manager

kuadrant-system

multus

dns-operator-controller-manager-65b49595d7-hphfv

AddedInterface

Add eth0 [10.132.0.26/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-operator-6d85f6564-xdn96

Created

Created container: manager

kuadrant-system

kubelet

dns-operator-controller-manager-65b49595d7-hphfv

Started

Started container manager

kuadrant-system

kubelet

dns-operator-controller-manager-65b49595d7-hphfv

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:ba46f54446ecd3ed1a3d877dcbb7359ff8f2d918827f3411a5cb0781915a65d5" in 1.785s (1.785s including waiting). Image size: 190472853 bytes.

kuadrant-system

kubelet

dns-operator-controller-manager-65b49595d7-hphfv

Created

Created container: manager

kuadrant-system

replicaset-controller

kuadrant-console-plugin-85fd4b8b9d

SuccessfulCreate

Created pod: kuadrant-console-plugin-85fd4b8b9d-4tc92

kuadrant-system

dns-operator-controller-manager-65b49595d7-hphfv_63057943-3655-4430-98d2-c1195706f4a7

a3f98d6c.kuadrant.io

LeaderElection

dns-operator-controller-manager-65b49595d7-hphfv_63057943-3655-4430-98d2-c1195706f4a7 became leader

kuadrant-system

deployment-controller

kuadrant-console-plugin

ScalingReplicaSet

Scaled up replica set kuadrant-console-plugin-85fd4b8b9d from 0 to 1

kuadrant-system

kubelet

kuadrant-console-plugin-85fd4b8b9d-4tc92

FailedMount

MountVolume.SetUp failed for volume "plugin-serving-cert" : secret "plugin-serving-cert" not found

kuadrant-system

multus

kuadrant-console-plugin-85fd4b8b9d-4tc92

AddedInterface

Add eth0 [10.134.0.35/23] from ovn-kubernetes

kuadrant-system

kubelet

kuadrant-console-plugin-85fd4b8b9d-4tc92

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:6e69f41d3d70b0afa7691a5d2400065d3bcdec2ea82830cf2992b8e69ba06e2a"

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.0

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kubelet

kuadrant-console-plugin-85fd4b8b9d-4tc92

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:6e69f41d3d70b0afa7691a5d2400065d3bcdec2ea82830cf2992b8e69ba06e2a" in 4.628s (4.628s including waiting). Image size: 342502139 bytes.

kuadrant-system

kubelet

kuadrant-console-plugin-85fd4b8b9d-4tc92

Created

Created container: kuadrant-console-plugin

kuadrant-system

kubelet

kuadrant-console-plugin-85fd4b8b9d-4tc92

Started

Started container kuadrant-console-plugin

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.1

InstallSucceeded

install strategy completed with no errors

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.1

InstallSucceeded

install strategy completed with no errors

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.0

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kubelet

limitador-limitador-ff68ff5cb-4ph8z

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:0b02c000d9a175235223ec5916345a1ccedeedc25c87cc0638e8554e6c318474"

kuadrant-system

multus

limitador-limitador-ff68ff5cb-4ph8z

AddedInterface

Add eth0 [10.132.0.27/23] from ovn-kubernetes

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-69574b596d from 0 to 1

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-ff68ff5cb from 0 to 1

kuadrant-system

replicaset-controller

limitador-limitador-69574b596d

SuccessfulCreate

Created pod: limitador-limitador-69574b596d-hnvp6

kuadrant-system

replicaset-controller

limitador-limitador-ff68ff5cb

SuccessfulCreate

Created pod: limitador-limitador-ff68ff5cb-4ph8z

kuadrant-system

replicaset-controller

authorino-8df697c9f

SuccessfulCreate

Created pod: authorino-8df697c9f-brb5n

kuadrant-system

multus

authorino-b6c89bb85-knflr

AddedInterface

Add eth0 [10.133.0.29/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-b6c89bb85-knflr

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a"

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-8df697c9f from 0 to 1

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-b6c89bb85 from 0 to 1

kuadrant-system

multus

limitador-limitador-69574b596d-hnvp6

AddedInterface

Add eth0 [10.134.0.36/23] from ovn-kubernetes

kuadrant-system

multus

authorino-8df697c9f-brb5n

AddedInterface

Add eth0 [10.133.0.30/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-8df697c9f-brb5n

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a"

kuadrant-system

replicaset-controller

authorino-b6c89bb85

SuccessfulCreate

Created pod: authorino-b6c89bb85-knflr

kuadrant-system

kubelet

authorino-8df697c9f-brb5n

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a" in 2.56s (2.56s including waiting). Image size: 202706732 bytes.

kuadrant-system

kubelet

authorino-b6c89bb85-knflr

Created

Created container: authorino

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-b6c89bb85 from 1 to 0

kuadrant-system

replicaset-controller

authorino-b6c89bb85

SuccessfulDelete

Deleted pod: authorino-b6c89bb85-knflr

kuadrant-system

kubelet

authorino-b6c89bb85-knflr

Started

Started container authorino

kuadrant-system

kubelet

authorino-b6c89bb85-knflr

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a" in 2.82s (2.82s including waiting). Image size: 202706732 bytes.

kuadrant-system

kubelet

authorino-8df697c9f-brb5n

Created

Created container: authorino

kuadrant-system

kubelet

authorino-8df697c9f-brb5n

Started

Started container authorino

kuadrant-system

kubelet

authorino-b6c89bb85-knflr

Killing

Stopping container authorino

kuadrant-system

kubelet

limitador-limitador-ff68ff5cb-4ph8z

Started

Started container limitador

kuadrant-system

kubelet

limitador-limitador-ff68ff5cb-4ph8z

Created

Created container: limitador

kuadrant-system

kubelet

limitador-limitador-ff68ff5cb-4ph8z

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:0b02c000d9a175235223ec5916345a1ccedeedc25c87cc0638e8554e6c318474" in 45.414s (45.414s including waiting). Image size: 250499765 bytes.

kuadrant-system

kubelet

limitador-limitador-69574b596d-hnvp6

Failed

Failed to pull image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:0b02c000d9a175235223ec5916345a1ccedeedc25c87cc0638e8554e6c318474": unable to pull image or OCI artifact: pull image err: copying system image from manifest list: reading signatures: reading signature from https://registry.redhat.io/containers/sigstore/rhcl-1/limitador-rhel9@sha256=e40f1438524bd554d9e23b0ddbaaa4ccf42d91ee5912ebdb372d11f734b9b524/signature-5: received unexpected HTTP status: 500 Internal Server Error; artifact err: provided artifact is a container image

kuadrant-system

kubelet

limitador-limitador-69574b596d-hnvp6

Failed

Error: ErrImagePull

kuadrant-system

kubelet

limitador-limitador-69574b596d-hnvp6

Failed

Error: ImagePullBackOff

kuadrant-system

kubelet

limitador-limitador-69574b596d-hnvp6

BackOff

Back-off pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:0b02c000d9a175235223ec5916345a1ccedeedc25c87cc0638e8554e6c318474"
(x2)

kuadrant-system

kubelet

limitador-limitador-69574b596d-hnvp6

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:0b02c000d9a175235223ec5916345a1ccedeedc25c87cc0638e8554e6c318474"

kuadrant-system

kubelet

limitador-limitador-69574b596d-hnvp6

Started

Started container limitador

kuadrant-system

kubelet

limitador-limitador-69574b596d-hnvp6

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:0b02c000d9a175235223ec5916345a1ccedeedc25c87cc0638e8554e6c318474" in 4.41s (4.41s including waiting). Image size: 250499765 bytes.

kuadrant-system

kubelet

limitador-limitador-69574b596d-hnvp6

Created

Created container: limitador

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled down replica set limitador-limitador-ff68ff5cb from 1 to 0

kuadrant-system

kubelet

limitador-limitador-ff68ff5cb-4ph8z

Killing

Stopping container limitador

kuadrant-system

replicaset-controller

limitador-limitador-ff68ff5cb

SuccessfulDelete

Deleted pod: limitador-limitador-ff68ff5cb-4ph8z

kuadrant-system

multus

authorino-7c77957575-cfthr

AddedInterface

Add eth0 [10.133.0.31/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-7c77957575-cfthr

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a"

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-7c77957575 from 0 to 1

kuadrant-system

replicaset-controller

authorino-7c77957575

SuccessfulCreate

Created pod: authorino-7c77957575-cfthr

kuadrant-system

kubelet

authorino-7c77957575-cfthr

Started

Started container authorino

kuadrant-system

kubelet

authorino-7c77957575-cfthr

Created

Created container: authorino

kuadrant-system

kubelet

authorino-7c77957575-cfthr

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a" in 465ms (465ms including waiting). Image size: 202706732 bytes.

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-8df697c9f from 1 to 0

kuadrant-system

kubelet

authorino-8df697c9f-brb5n

Killing

Stopping container authorino

kuadrant-system

replicaset-controller

authorino-8df697c9f

SuccessfulDelete

Deleted pod: authorino-8df697c9f-brb5n

kserve-ci-e2e-test

multus

router-gateway-1-openshift-default-799f46c59b-kkqwb

AddedInterface

Add eth0 [10.132.0.31/23] from ovn-kubernetes

kserve-ci-e2e-test

service-controller

router-gateway-1-openshift-default

EnsuringLoadBalancer

Ensuring load balancer

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-799f46c59b

SuccessfulCreate

Created pod: router-gateway-1-openshift-default-799f46c59b-kkqwb

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-54c47cdfc7

SuccessfulCreate

Created pod: router-gateway-1-openshift-default-54c47cdfc7-lfprg

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-kkqwb

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8"

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-lfprg

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8"

kserve-ci-e2e-test

multus

router-gateway-1-openshift-default-54c47cdfc7-lfprg

AddedInterface

Add eth0 [10.132.0.32/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-kkqwb

Started

Started container istio-proxy

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-kkqwb

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" in 3.706s (3.706s including waiting). Image size: 181251556 bytes.

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-kkqwb

Created

Created container: istio-proxy

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-lfprg

Started

Started container istio-proxy

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-lfprg

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" in 3.513s (3.513s including waiting). Image size: 181251556 bytes.

kserve-ci-e2e-test

service-controller

router-gateway-1-openshift-default

EnsuredLoadBalancer

Ensured load balancer

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-lfprg

Created

Created container: istio-proxy

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-799f46c59b

SuccessfulDelete

Deleted pod: router-gateway-1-openshift-default-799f46c59b-kkqwb

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.HTTPRoute kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-route

kserve-ci-e2e-test

replicaset-controller

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc5

SuccessfulCreate

Created pod: gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc5cktt6

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.Secret kserve-ci-e2e-test/gw-sec0c69dceeb48768325d1a53a749e65786-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.Service kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.Deployment kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve

kserve-ci-e2e-test

deployment-controller

gw-section-name-router-with-gat-2f0a622e-kserve

ScalingReplicaSet

Scaled up replica set gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc5 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-2f0a622e

Created

Created v1.DestinationRule kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-workload-svc

kserve-ci-e2e-test

multus

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc5cktt6

AddedInterface

Add eth0 [10.133.0.32/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc5cktt6

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-kkqwb

Killing

Stopping container istio-proxy

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc5cktt6

Created

Created container: main

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc5cktt6

Started

Started container main

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-799f46c59b

SuccessfulCreate

Created pod: router-gateway-1-openshift-default-799f46c59b-fqc69

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc5cktt6

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.206s (1.206s including waiting). Image size: 98346788 bytes.

kserve-ci-e2e-test

multus

router-gateway-1-openshift-default-799f46c59b-fqc69

AddedInterface

Add eth0 [10.134.0.38/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-54c47cdfc7

SuccessfulDelete

Deleted pod: router-gateway-1-openshift-default-54c47cdfc7-lfprg

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-lfprg

Killing

Stopping container istio-proxy
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

precise-prefix-cache-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy precise-prefix-cache-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/precise-prefix-cache-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "precise-prefix-cache-test-kserve-route" not found

kserve-ci-e2e-test

deployment-controller

precise-prefix-cache-test-kserve

ScalingReplicaSet

Scaled up replica set precise-prefix-cache-test-kserve-6ffc955574 from 0 to 2

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Secret kserve-ci-e2e-test/precise-prefix-cache-test-kserve-self-signed-certs

kserve-ci-e2e-test

replicaset-controller

precise-prefix-cache-test-kserve-6ffc955574

SuccessfulCreate

Created pod: precise-prefix-cache-test-kserve-6ffc955574-v494f

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/precise-prefix-cache-test-epp-sa

kserve-ci-e2e-test

replicaset-controller

precise-prefix-cache-test-kserve-6ffc955574

SuccessfulCreate

Created pod: precise-prefix-cache-test-kserve-6ffc955574-p26g2

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Deployment kserve-ci-e2e-test/precise-prefix-cache-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Service kserve-ci-e2e-test/precise-prefix-cache-test-kserve-workload-svc

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-d84595fbj5ftw

Pulling

Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1"

kserve-ci-e2e-test

multus

precise-prefix-cache-test-kserve-6ffc955574-v494f

AddedInterface

Add eth0 [10.132.0.33/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Service kserve-ci-e2e-test/precise-prefix-cache-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Deployment kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-6ffc955574-p26g2

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-6ffc955574-p26g2

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-6ffc955574-p26g2

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/precise-prefix-cache-test-epp-rb

kserve-ci-e2e-test

multus

precise-prefix-cache-test-kserve-router-scheduler-d84595fbj5ftw

AddedInterface

Add eth0 [10.132.0.34/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-6ffc955574-v494f

Pulling

Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1"

kserve-ci-e2e-test

replicaset-controller

precise-prefix-cache-test-kserve-router-scheduler-d84595fb

SuccessfulCreate

Created pod: precise-prefix-cache-test-kserve-router-scheduler-d84595fbj5ftw

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.Role kserve-ci-e2e-test/precise-prefix-cache-test-epp-role

kserve-ci-e2e-test

multus

precise-prefix-cache-test-kserve-6ffc955574-p26g2

AddedInterface

Add eth0 [10.134.0.39/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-precise-prefix-cache-test-epp-auth-rb

kserve-ci-e2e-test

deployment-controller

precise-prefix-cache-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set precise-prefix-cache-test-kserve-router-scheduler-d84595fb from 0 to 1

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc5cktt6

Killing

Stopping container main
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-fqc69

Created

Created container: istio-proxy
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-fqc69

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-fqc69

Started

Started container istio-proxy

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-6ffc955574-p26g2

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-6ffc955574-v494f

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-6ffc955574-v494f

Pulled

Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" in 5.156s (5.156s including waiting). Image size: 300570036 bytes.

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-d84595fbj5ftw

Pulled

Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" in 4.75s (4.75s including waiting). Image size: 300570036 bytes.

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-d84595fbj5ftw

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-d84595fbj5ftw

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-6ffc955574-p26g2

Started

Started container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-6ffc955574-p26g2

Created

Created container: main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-6ffc955574-p26g2

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.48s (1.48s including waiting). Image size: 98346788 bytes.

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-6ffc955574-v494f

Created

Created container: storage-initializer

kserve-ci-e2e-test

multus

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b4jkrn

AddedInterface

Add eth0 [10.134.0.40/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

gw-section-name-router-with-gat-f1d92d0f-kserve

ScalingReplicaSet

Scaled up replica set gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.HTTPRoute kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.DestinationRule kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.Service kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.Secret kserve-ci-e2e-test/gw-sec2774c263d49959f50d9eebc552e13bf9-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-d84595fbj5ftw

Pulling

Pulling image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2"

kserve-ci-e2e-test

LLMInferenceServiceController

gw-section-name-router-with-gat-f1d92d0f

Created

Created v1.Deployment kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve

kserve-ci-e2e-test

replicaset-controller

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b

SuccessfulCreate

Created pod: gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b4jkrn

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b4jkrn

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b4jkrn

Created

Created container: main

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b4jkrn

Started

Started container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-d84595fbj5ftw

Pulling

Pulling image "ghcr.io/llm-d/llm-d-uds-tokenizer:vllm-v0.19.1"

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-d84595fbj5ftw

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" in 1.694s (1.694s including waiting). Image size: 75073927 bytes.

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-54c47cdfc7

SuccessfulCreate

Created pod: router-gateway-1-openshift-default-54c47cdfc7-kgtbh

kserve-ci-e2e-test

multus

router-gateway-1-openshift-default-54c47cdfc7-kgtbh

AddedInterface

Add eth0 [10.133.0.33/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-6ffc955574-v494f

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-kgtbh

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8"

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-kgtbh

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" in 2.49s (2.49s including waiting). Image size: 181251556 bytes.

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-6ffc955574-v494f

Started

Started container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-6ffc955574-v494f

Created

Created container: main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-6ffc955574-v494f

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.796s (1.796s including waiting). Image size: 98346788 bytes.

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-655f7c58b4-8g97n

ProbeError

Readiness probe error: Get "http://10.134.0.33:15021/healthz/ready": read tcp 10.134.0.2:47062->10.134.0.33:15021: read: connection reset by peer body:

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-655f7c58b4-8g97n

Unhealthy

Readiness probe failed: Get "http://10.134.0.33:15021/healthz/ready": read tcp 10.134.0.2:47062->10.134.0.33:15021: read: connection reset by peer

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-fqc69

Killing

Stopping container istio-proxy

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-799f46c59b

SuccessfulDelete

Deleted pod: router-gateway-1-openshift-default-799f46c59b-fqc69

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/precise-prefix-cache-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Updated

Updated v1.Secret kserve-ci-e2e-test/precise-prefix-cache-test-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/precise-prefix-cache-test-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b4jkrn

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-d84595fbj5ftw

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-uds-tokenizer:vllm-v0.19.1" in 30.148s (30.148s including waiting). Image size: 2989890188 bytes.
(x2)

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-d84595fbj5ftw

Started

Started container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-d84595fbj5ftw

Started

Started container tokenizer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-d84595fbj5ftw

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-d84595fbj5ftw

Created

Created container: tokenizer
(x2)

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-d84595fbj5ftw

Created

Created container: main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-d84595fbj5ftw

Unhealthy

Liveness probe failed: timeout: failed to connect service "10.132.0.34:9003" within 1s: context deadline exceeded

kserve-ci-e2e-test

kubelet

gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b4jkrn

Unhealthy

Readiness probe failed: Get "https://10.134.0.40:8000/health": EOF

kserve-ci-e2e-test

deployment-controller

auth-enabled-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set auth-enabled-test-kserve-router-scheduler-7b8b8fbfcb from 0 to 1

kserve-ci-e2e-test

replicaset-controller

auth-enabled-test-kserve-router-scheduler-7b8b8fbfcb

SuccessfulCreate

Created pod: auth-enabled-test-kserve-router-scheduler-7b8b8fbfcb-h298k

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.Deployment kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler

kserve-ci-e2e-test

deployment-controller

auth-enabled-test-kserve

ScalingReplicaSet

Scaled up replica set auth-enabled-test-kserve-7dfddf4646 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.Deployment kserve-ci-e2e-test/auth-enabled-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.Secret kserve-ci-e2e-test/auth-enabled-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.Service kserve-ci-e2e-test/auth-enabled-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/auth-enabled-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-enabled-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.Role kserve-ci-e2e-test/auth-enabled-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.Service kserve-ci-e2e-test/auth-enabled-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/auth-enabled-test-epp-rb

kserve-ci-e2e-test

replicaset-controller

auth-enabled-test-kserve-7dfddf4646

SuccessfulCreate

Created pod: auth-enabled-test-kserve-7dfddf4646-c84nz

kserve-ci-e2e-test

multus

auth-enabled-test-kserve-7dfddf4646-c84nz

AddedInterface

Add eth0 [10.134.0.41/23] from ovn-kubernetes

kserve-ci-e2e-test

multus

auth-enabled-test-kserve-router-scheduler-7b8b8fbfcb-h298k

AddedInterface

Add eth0 [10.134.0.42/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-7b8b8fbfcb-h298k

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-7b8b8fbfcb-h298k

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-7b8b8fbfcb-h298k

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-7dfddf4646-c84nz

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-7dfddf4646-c84nz

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-7dfddf4646-c84nz

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-7b8b8fbfcb-h298k

Pulling

Pulling image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2"

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-7dfddf4646-c84nz

Pulling

Pulling image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0"

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-7b8b8fbfcb-h298k

Started

Started container main

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-7b8b8fbfcb-h298k

Created

Created container: main

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-7b8b8fbfcb-h298k

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" in 1.571s (1.571s including waiting). Image size: 75073927 bytes.
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/auth-enabled-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Updated

Updated v1.Secret kserve-ci-e2e-test/auth-enabled-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/auth-enabled-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

LLMInferenceServiceReady

LLMInferenceService [precise-prefix-cache-test] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

precise-prefix-cache-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-precise-prefix-cache-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-d84595fbj5ftw

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-d84595fbj5ftw

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-6ffc955574-p26g2

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-router-scheduler-d84595fbj5ftw

Killing

Stopping container tokenizer

kserve-ci-e2e-test

kubelet

precise-prefix-cache-test-kserve-6ffc955574-v494f

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-7dfddf4646-c84nz

Pulled

Successfully pulled image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" in 27.071s (27.071s including waiting). Image size: 3531177328 bytes.

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-7dfddf4646-c84nz

Created

Created container: main

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-7dfddf4646-c84nz

Started

Started container main
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-router-managed-test-llm-e45d1f79

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-router-managed-test-llm-e45d1f79-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-router-managed-test-llm-e45d1f79-kserve-route" not found

kserve-ci-e2e-test

replicaset-controller

llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879

SuccessfulCreate

Created pod: llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879ntk8x

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-role

kserve-ci-e2e-test

deployment-controller

llmisvc-router-managed-test-llm-e45d1f79-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879 from 0 to 1

kserve-ci-e2e-test

kubelet

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schel7mgq

Pulling

Pulling image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2"

kserve-ci-e2e-test

multus

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schel7mgq

AddedInterface

Add eth0 [10.133.0.34/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879ntk8x

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879ntk8x

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879ntk8x

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve

kserve-ci-e2e-test

multus

llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879ntk8x

AddedInterface

Add eth0 [10.132.0.35/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-sa

kserve-ci-e2e-test

deployment-controller

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheduler-5dd88bfbb7 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheduler-5dd88bfbb7

SuccessfulCreate

Created pod: llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schel7mgq

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-testef4d2875be14b30dc1561ed84d0d4bde-llmisvc-rout

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.Secret kserve-ci-e2e-test/llmisv5c7e67b6c51568d1d6d13829a9337f2a-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-rb

kserve-ci-e2e-test

kubelet

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schel7mgq

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" in 1.205s (1.205s including waiting). Image size: 75073927 bytes.

kserve-ci-e2e-test

kubelet

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schel7mgq

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schel7mgq

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv5c7e67b6c51568d1d6d13829a9337f2a-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

LLMInferenceServiceReady

LLMInferenceService [llmisvc-router-managed-test-llm-e45d1f79] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-e45d1f79

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-testef4d2875be14b30dc1561ed84d0d4bde-llmisvc-rout

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879ntk8x

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schel7mgq

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schel7mgq

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879ntk8x

Unhealthy

Readiness probe failed: Get "https://10.132.0.35:8000/health": dial tcp 10.132.0.35:8000: connect: connection refused
(x11)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-router-managed-test-llm-5b1e8f15

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-router-managed-test-llm-5b1e8f15-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-router-managed-test-llm-5b1e8f15-kserve-route" not found

kserve-ci-e2e-test

replicaset-controller

llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4

SuccessfulCreate

Created pod: llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4jt7jj

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.Secret kserve-ci-e2e-test/llmisve55ae740357a3a31a27cdb8b66ffe20f-kserve-self-signed-certs

kserve-ci-e2e-test

deployment-controller

llmisvc-router-managed-test-llm-5b1e8f15-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4 from 0 to 1

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4jt7jj

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test7f54e84970003a6e7372bdbcb574f7ed-llmisvc-rout

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-sa

kserve-ci-e2e-test

replicaset-controller

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheduler-749449dbc8

SuccessfulCreate

Created pod: llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schek22wf

kserve-ci-e2e-test

deployment-controller

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheduler-749449dbc8 from 0 to 1

kserve-ci-e2e-test

multus

llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4jt7jj

AddedInterface

Add eth0 [10.132.0.36/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4jt7jj

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4jt7jj

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve

kserve-ci-e2e-test

kubelet

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schek22wf

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schek22wf

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schek22wf

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

multus

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schek22wf

AddedInterface

Add eth0 [10.133.0.35/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisve55ae740357a3a31a27cdb8b66ffe20f-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-shadow-svc
(x10)

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-7dfddf4646-c84nz

Unhealthy

Startup probe failed: Get "https://10.134.0.41:8000/health": dial tcp 10.134.0.41:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

LLMInferenceServiceReady

LLMInferenceService [auth-enabled-test] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

auth-enabled-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-enabled-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-7b8b8fbfcb-h298k

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-router-scheduler-7b8b8fbfcb-h298k

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-7dfddf4646-c84nz

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

LLMInferenceServiceReady

LLMInferenceService [llmisvc-router-managed-test-llm-5b1e8f15] is Ready
(x11)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-service

kserve-ci-e2e-test

deployment-controller

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheduler-6d86bd4d9d from 0 to 1

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58dd

SuccessfulCreate

Created pod: llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddcrqxt

kserve-ci-e2e-test

replicaset-controller

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheduler-6d86bd4d9d

SuccessfulCreate

Created pod: llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche7bbzq

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-testb7025ca4d8a6f8f5b2fd08b5581d2678-llmisvc-mode

kserve-ci-e2e-test

deployment-controller

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58dd from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.Secret kserve-ci-e2e-test/llmisvb11a9c9832b99b016bc8f8e0ea095712-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-workload-svc

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddcrqxt

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddcrqxt

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche7bbzq

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche7bbzq

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddcrqxt

Created

Created container: main

kserve-ci-e2e-test

multus

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddcrqxt

AddedInterface

Add eth0 [10.133.0.36/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche7bbzq

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

multus

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche7bbzq

AddedInterface

Add eth0 [10.132.0.37/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

auth-enabled-test-kserve-7dfddf4646-c84nz

Unhealthy

Readiness probe failed: Get "https://10.134.0.41:8000/health": context deadline exceeded

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisvb11a9c9832b99b016bc8f8e0ea095712-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.Secret kserve-ci-e2e-test/auth-invalid-token-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.Deployment kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler

kserve-ci-e2e-test

replicaset-controller

auth-invalid-token-test-kserve-router-scheduler-6c486c5958

SuccessfulCreate

Created pod: auth-invalid-token-test-kserve-router-scheduler-6c486c59585thrg

kserve-ci-e2e-test

deployment-controller

auth-invalid-token-test-kserve

ScalingReplicaSet

Scaled up replica set auth-invalid-token-test-kserve-58b5cf756b from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.Service kserve-ci-e2e-test/auth-invalid-token-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/auth-invalid-token-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-invalid-token-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.Role kserve-ci-e2e-test/auth-invalid-token-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.Service kserve-ci-e2e-test/auth-invalid-token-test-epp-service

kserve-ci-e2e-test

deployment-controller

auth-invalid-token-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set auth-invalid-token-test-kserve-router-scheduler-6c486c5958 from 0 to 1

kserve-ci-e2e-test

multus

auth-invalid-token-test-kserve-58b5cf756b-27d2s

AddedInterface

Add eth0 [10.134.0.43/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-58b5cf756b-27d2s

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-58b5cf756b-27d2s

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-58b5cf756b-27d2s

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.Deployment kserve-ci-e2e-test/auth-invalid-token-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/auth-invalid-token-test-epp-rb

kserve-ci-e2e-test

replicaset-controller

auth-invalid-token-test-kserve-58b5cf756b

SuccessfulCreate

Created pod: auth-invalid-token-test-kserve-58b5cf756b-27d2s

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-6c486c59585thrg

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-6c486c59585thrg

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-6c486c59585thrg

Started

Started container storage-initializer

kserve-ci-e2e-test

multus

auth-invalid-token-test-kserve-router-scheduler-6c486c59585thrg

AddedInterface

Add eth0 [10.134.0.44/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-6c486c59585thrg

Created

Created container: main

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-6c486c59585thrg

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-6c486c59585thrg

Started

Started container main
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-58b5cf756b-27d2s

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-58b5cf756b-27d2s

Created

Created container: main

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-58b5cf756b-27d2s

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Updated

Updated v1.Secret kserve-ci-e2e-test/auth-invalid-token-test-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/auth-invalid-token-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/auth-invalid-token-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-qwen2-5-0-5b-rout-a50492e9] is Ready

kserve-ci-e2e-test

kubelet

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche7bbzq

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche7bbzq

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-a50492e9

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-testb7025ca4d8a6f8f5b2fd08b5581d2678-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddcrqxt

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-5b1e8f15

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test7f54e84970003a6e7372bdbcb574f7ed-llmisvc-rout

kserve-ci-e2e-test

kubelet

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schek22wf

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4jt7jj

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schek22wf

FailedPreStopHook

PreStopHook failed
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-router-managed-test-llm-4b931143

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-router-managed-test-llm-4b931143-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-router-managed-test-llm-4b931143-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test8ac8e3d2264ccb939eb021b0b835847c-llmisvc-rout

kserve-ci-e2e-test

replicaset-controller

llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56

SuccessfulCreate

Created pod: llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-fwwrx

kserve-ci-e2e-test

deployment-controller

llmisvc-router-managed-test-llm-4b931143-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56 from 0 to 1

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-fwwrx

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.Secret kserve-ci-e2e-test/llmisvca2d2d7d499abb359505529ebe02c136-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-workload-svc

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-fwwrx

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-role

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-fwwrx

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-service

kserve-ci-e2e-test

multus

llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-fwwrx

AddedInterface

Add eth0 [10.132.0.38/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-scheduler-68cc9685d6 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-scheduler-68cc9685d6

SuccessfulCreate

Created pod: llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schex5jnk

kserve-ci-e2e-test

kubelet

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schex5jnk

Started

Started container main

kserve-ci-e2e-test

multus

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schex5jnk

AddedInterface

Add eth0 [10.133.0.37/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schex5jnk

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schex5jnk

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisvca2d2d7d499abb359505529ebe02c136-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

LLMInferenceServiceReady

LLMInferenceService [llmisvc-router-managed-test-llm-4b931143] is Ready
(x11)

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-58b5cf756b-27d2s

Unhealthy

Startup probe failed: Get "https://10.134.0.43:8000/health": dial tcp 10.134.0.43:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

LLMInferenceServiceReady

LLMInferenceService [auth-invalid-token-test] is Ready

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-58b5cf756b-27d2s

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

auth-invalid-token-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-invalid-token-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-6c486c59585thrg

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-router-scheduler-6c486c59585thrg

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

auth-invalid-token-test-kserve-58b5cf756b-27d2s

Unhealthy

Readiness probe failed: Get "https://10.134.0.43:8000/health": dial tcp 10.134.0.43:8000: connect: connection refused
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

auth-disabled-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy auth-disabled-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/auth-disabled-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "auth-disabled-test-kserve-route" not found

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-ccd54d6d4-cgwk9

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-ccd54d6d4-cgwk9

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.Deployment kserve-ci-e2e-test/auth-disabled-test-kserve

kserve-ci-e2e-test

deployment-controller

auth-disabled-test-kserve

ScalingReplicaSet

Scaled up replica set auth-disabled-test-kserve-ccd54d6d4 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.Deployment kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler

kserve-ci-e2e-test

deployment-controller

auth-disabled-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set auth-disabled-test-kserve-router-scheduler-58f5ddbc8f from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.Service kserve-ci-e2e-test/auth-disabled-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.Service kserve-ci-e2e-test/auth-disabled-test-epp-service

kserve-ci-e2e-test

replicaset-controller

auth-disabled-test-kserve-router-scheduler-58f5ddbc8f

SuccessfulCreate

Created pod: auth-disabled-test-kserve-router-scheduler-58f5ddbc8f-qgm5g

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.Role kserve-ci-e2e-test/auth-disabled-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-disabled-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/auth-disabled-test-epp-rb

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-ccd54d6d4-cgwk9

Created

Created container: storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.Secret kserve-ci-e2e-test/auth-disabled-test-kserve-self-signed-certs

kserve-ci-e2e-test

multus

auth-disabled-test-kserve-ccd54d6d4-cgwk9

AddedInterface

Add eth0 [10.134.0.45/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/auth-disabled-test-epp-sa

kserve-ci-e2e-test

replicaset-controller

auth-disabled-test-kserve-ccd54d6d4

SuccessfulCreate

Created pod: auth-disabled-test-kserve-ccd54d6d4-cgwk9

kserve-ci-e2e-test

multus

auth-disabled-test-kserve-router-scheduler-58f5ddbc8f-qgm5g

AddedInterface

Add eth0 [10.132.0.39/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-router-scheduler-58f5ddbc8f-qgm5g

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-router-scheduler-58f5ddbc8f-qgm5g

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-router-scheduler-58f5ddbc8f-qgm5g

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-router-scheduler-58f5ddbc8f-qgm5g

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-router-scheduler-58f5ddbc8f-qgm5g

Created

Created container: main

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-router-scheduler-58f5ddbc8f-qgm5g

Started

Started container main

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-ccd54d6d4-cgwk9

Started

Started container main

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-ccd54d6d4-cgwk9

Created

Created container: main

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-ccd54d6d4-cgwk9

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-workload-svc

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fb2h9x

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test76d7f532acb694e4a7bcef75d32cd8a1-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fb2h9x

Started

Started container main

kserve-ci-e2e-test

deployment-controller

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f from 0 to 1

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fb2h9x

Created

Created container: main

kserve-ci-e2e-test

multus

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schecjg9g

AddedInterface

Add eth0 [10.134.0.46/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-rb

kserve-ci-e2e-test

multus

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fb2h9x

AddedInterface

Add eth0 [10.133.0.38/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f

SuccessfulCreate

Created pod: llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fb2h9x

kserve-ci-e2e-test

replicaset-controller

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheduler-6fcb489785

SuccessfulCreate

Created pod: llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schecjg9g

kserve-ci-e2e-test

deployment-controller

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheduler-6fcb489785 from 0 to 1

kserve-ci-e2e-test

kubelet

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schecjg9g

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.Secret kserve-ci-e2e-test/llmisv08544b88a8d963ffd553cc1f3ed82d16-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-service

kserve-ci-e2e-test

kubelet

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schecjg9g

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schecjg9g

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/auth-disabled-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Updated

Updated v1.Secret kserve-ci-e2e-test/auth-disabled-test-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/auth-disabled-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv08544b88a8d963ffd553cc1f3ed82d16-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-qwen2-5-0-5b-rout-4f8c0978] is Ready

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fb2h9x

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test76d7f532acb694e4a7bcef75d32cd8a1-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schecjg9g

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schecjg9g

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fb2h9x

Unhealthy

Readiness probe failed: Get "https://10.133.0.38:8000/health": dial tcp 10.133.0.38:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schex5jnk

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-fwwrx

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schex5jnk

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-router-managed-test-llm-4b931143

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test8ac8e3d2264ccb939eb021b0b835847c-llmisvc-rout

kserve-ci-e2e-test

kubelet

llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-fwwrx

Unhealthy

Readiness probe failed: Get "https://10.132.0.38:8000/health": dial tcp 10.132.0.38:8000: connect: connection refused

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-with-7ca60146-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-with-7ca60146-kserve-79c7c55747 from 0 to 1
(x11)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-with-7ca60146

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.Secret kserve-ci-e2e-test/llmisv3e414c2ba058a022dfd694dbcbac5b51-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-79c7c55747

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-with-7ca60146-kserve-79c7c55747-lfmdf

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-79c7c55747-lfmdf

Created

Created container: storage-initializer

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scher2z97

AddedInterface

Add eth0 [10.132.0.40/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheduler-6698955b74 from 0 to 1

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-79c7c55747-lfmdf

AddedInterface

Add eth0 [10.134.0.47/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-workload-svc

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-79c7c55747-lfmdf

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test5216bfd716f919dc046bc693ceb22e41-llmisvc-mode

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-rb

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-79c7c55747-lfmdf

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-service

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheduler-6698955b74

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scher2z97

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-sa

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scher2z97

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scher2z97

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scher2z97

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scher2z97

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scher2z97

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scher2z97

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-79c7c55747-lfmdf

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-79c7c55747-lfmdf

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-79c7c55747-lfmdf

Created

Created container: main
(x11)

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-ccd54d6d4-cgwk9

Unhealthy

Startup probe failed: Get "https://10.134.0.45:8000/health": dial tcp 10.134.0.45:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv3e414c2ba058a022dfd694dbcbac5b51-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

LLMInferenceServiceReady

LLMInferenceService [auth-disabled-test] is Ready

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-router-scheduler-58f5ddbc8f-qgm5g

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

auth-disabled-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-disabled-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-ccd54d6d4-cgwk9

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-router-scheduler-58f5ddbc8f-qgm5g

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

auth-disabled-test-kserve-ccd54d6d4-cgwk9

Unhealthy

Readiness probe failed: Get "https://10.134.0.45:8000/health": dial tcp 10.134.0.45:8000: connect: connection refused

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-799f46c59b

SuccessfulCreate

Created pod: router-gateway-1-openshift-default-799f46c59b-s6nj7

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-s6nj7

Started

Started container istio-proxy

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-s6nj7

Created

Created container: istio-proxy

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-s6nj7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-655f7c58b4-8g97n

BackOff

Back-off restarting failed container istio-proxy in pod openshift-ai-inference-openshift-default-655f7c58b4-8g97n_openshift-ingress(87a48a7a-6d63-43f0-8bab-7e29c1a6979a)

kserve-ci-e2e-test

multus

router-gateway-1-openshift-default-799f46c59b-s6nj7

AddedInterface

Add eth0 [10.133.0.39/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-799f46c59b

SuccessfulDelete

Deleted pod: router-gateway-1-openshift-default-799f46c59b-s6nj7

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-s6nj7

Killing

Stopping container istio-proxy
(x11)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-e95b1dc1

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-route" not found

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4ctbbm

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-e95b1dc1

Created

Created v1.Secret kserve-ci-e2e-test/llmisv122f03714c5bdf915a2917fdf1262b98-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-e95b1dc1

Created

Created v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-e95b1dc1

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-e95b1dc1

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4ctbbm

Created

Created container: main

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4ctbbm

AddedInterface

Add eth0 [10.133.0.40/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4ctbbm

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4ctbbm

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-e95b1dc1

Created

Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-workload-svc
(x3)

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-655f7c58b4-8g97n

Created

Created container: istio-proxy
(x3)

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-655f7c58b4-8g97n

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-655f7c58b4-8g97n

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-e95b1dc1

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv122f03714c5bdf915a2917fdf1262b98-kserve-self-signed-certs
(x11)

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-655f7c58b4-8g97n

Unhealthy

Startup probe failed: Get "http://10.134.0.33:15021/healthz/ready": dial tcp 10.134.0.33:15021: connect: connection refused
(x11)

openshift-ingress

kubelet

openshift-ai-inference-openshift-default-655f7c58b4-8g97n

ProbeError

Startup probe error: Get "http://10.134.0.33:15021/healthz/ready": dial tcp 10.134.0.33:15021: connect: connection refused body:

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-e95b1dc1

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-fb-opt-125m-route-e95b1dc1] is Ready
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-kgtbh

Created

Created container: istio-proxy
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-kgtbh

Started

Started container istio-proxy

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-kgtbh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
(x6)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-kgtbh

Unhealthy

Startup probe failed: Get "http://10.133.0.33:15021/healthz/ready": dial tcp 10.133.0.33:15021: connect: connection refused
(x8)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-79c7c55747-lfmdf

Unhealthy

Startup probe failed: Get "https://10.134.0.47:8000/health": dial tcp 10.134.0.47:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-fb-opt-125m-with-7ca60146] is Ready

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-79c7c55747-lfmdf

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-7ca60146

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test5216bfd716f919dc046bc693ceb22e41-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scher2z97

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scher2z97

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.Secret kserve-ci-e2e-test/llmisv77ff2528d3e9b4972cd9335229fce9f0-kserve-self-signed-certs
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-with-ba4d693a

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route" not found

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-794bb95d4d

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-794bb95d4d-s6j8f

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test05addb65ba05195619f26ef266e8fc04-llmisvc-mode

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-scheduler-844f8ff647

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schelnjhn

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-scheduler-844f8ff647 from 0 to 1

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-794bb95d4d-s6j8f

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-sa

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-794bb95d4d-s6j8f

AddedInterface

Add eth0 [10.134.0.48/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-rb

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-794bb95d4d-s6j8f

Created

Created container: storage-initializer

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-794bb95d4d from 0 to 1

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-794bb95d4d-s6j8f

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schelnjhn

AddedInterface

Add eth0 [10.132.0.41/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schelnjhn

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schelnjhn

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schelnjhn

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schelnjhn

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schelnjhn

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schelnjhn

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-794bb95d4d-s6j8f

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-794bb95d4d-s6j8f

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-794bb95d4d-s6j8f

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv77ff2528d3e9b4972cd9335229fce9f0-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4ctbbm

Killing

Stopping container main
(x10)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-87882a8e

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-87882a8e-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-87882a8e-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheduler

kserve-ci-e2e-test

replicaset-controller

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheduler-688ff6d544

SuccessfulCreate

Created pod: llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scherchj5

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.Secret kserve-ci-e2e-test/llmisva690bbc929faec8bc98c767f16c003c1-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5db6d9f47djrd27

Created

Created container: storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test21fe6730fe484f3a92b1a16afe1bac8f-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5db6d9f47djrd27

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-epp-role

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5db6d9f47d

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5db6d9f47djrd27

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5db6d9f47djrd27

AddedInterface

Add eth0 [10.134.0.49/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-87882a8e-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5db6d9f47d from 0 to 1

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5db6d9f47djrd27

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

deployment-controller

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheduler-688ff6d544 from 0 to 1

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scherchj5

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scherchj5

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scherchj5

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scherchj5

Created

Created container: main

kserve-ci-e2e-test

multus

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scherchj5

AddedInterface

Add eth0 [10.132.0.42/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scherchj5

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scherchj5

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine
(x7)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-794bb95d4d-s6j8f

Unhealthy

Startup probe failed: Get "https://10.134.0.48:8000/health": dial tcp 10.134.0.48:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5db6d9f47djrd27

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5db6d9f47djrd27

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5db6d9f47djrd27

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisva690bbc929faec8bc98c767f16c003c1-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-fb-opt-125m-with-ba4d693a] is Ready

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schelnjhn

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-with-ba4d693a

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test05addb65ba05195619f26ef266e8fc04-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schelnjhn

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-794bb95d4d-s6j8f

Killing

Stopping container main

kserve-ci-e2e-test

ebs.csi.aws.com_aws-ebs-csi-driver-controller-d8db6f98-dhbrp_42e66c8b-a583-47f3-9a2d-f39fd5be165e

e2e-pvc-model-storage

Provisioning

External provisioner is provisioning volume for claim "kserve-ci-e2e-test/e2e-pvc-model-storage"

kserve-ci-e2e-test

job-controller

e2e-pvc-model-download

SuccessfulCreate

Created pod: e2e-pvc-model-download-w2mvs

kserve-ci-e2e-test

persistentvolume-controller

e2e-pvc-model-storage

WaitForFirstConsumer

waiting for first consumer to be created before binding

kserve-ci-e2e-test

persistentvolume-controller

e2e-pvc-model-storage

ExternalProvisioning

Waiting for a volume to be created either by the external provisioner 'ebs.csi.aws.com' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered.

kserve-ci-e2e-test

ebs.csi.aws.com_aws-ebs-csi-driver-controller-d8db6f98-dhbrp_42e66c8b-a583-47f3-9a2d-f39fd5be165e

e2e-pvc-model-storage

ProvisioningSucceeded

Successfully provisioned volume pvc-3823dadc-7c52-412c-8ef9-84caab00a904

kserve-ci-e2e-test

attachdetach-controller

e2e-pvc-model-download-w2mvs

SuccessfulAttachVolume

AttachVolume.Attach succeeded for volume "pvc-3823dadc-7c52-412c-8ef9-84caab00a904"

kserve-ci-e2e-test

multus

e2e-pvc-model-download-w2mvs

AddedInterface

Add eth0 [10.132.0.43/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

e2e-pvc-model-download-w2mvs

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

e2e-pvc-model-download-w2mvs

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

e2e-pvc-model-download-w2mvs

Started

Started container storage-initializer

kserve-ci-e2e-test

job-controller

e2e-pvc-model-download

Completed

Job completed

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.Secret kserve-ci-e2e-test/llmisv8bf079eb6eda4debfb4ef5bb7817824c-kserve-self-signed-certs

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9d

SuccessfulCreate

Created pod: llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9dmww7f

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-testc9569cf4801efc0ed27b2f25ffaee875-llmisvc-mode

kserve-ci-e2e-test

deployment-controller

llmisvc-model-pvc-router-manage-59b9d263-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9d from 0 to 1
(x9)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-pvc-router-manage-59b9d263

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-pvc-router-manage-59b9d263-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-pvc-router-manage-59b9d263-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-workload-svc

kserve-ci-e2e-test

kubelet

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemgcff

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-epp-service

kserve-ci-e2e-test

replicaset-controller

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheduler-5f555d4d85

SuccessfulCreate

Created pod: llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemgcff

kserve-ci-e2e-test

deployment-controller

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheduler-5f555d4d85 from 0 to 1

kserve-ci-e2e-test

multus

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemgcff

AddedInterface

Add eth0 [10.133.0.41/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemgcff

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemgcff

Started

Started container main

kserve-ci-e2e-test

attachdetach-controller

llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemgcff

SuccessfulAttachVolume

AttachVolume.Attach succeeded for volume "pvc-3823dadc-7c52-412c-8ef9-84caab00a904"

kserve-ci-e2e-test

attachdetach-controller

llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9dmww7f

FailedAttachVolume

Multi-Attach error for volume "pvc-3823dadc-7c52-412c-8ef9-84caab00a904" Volume is already used by pod(s) llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemgcff

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-shadow-svc
(x11)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5db6d9f47djrd27

Unhealthy

Startup probe failed: Get "https://10.134.0.49:8000/health": dial tcp 10.134.0.49:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-fb-opt-125m-route-87882a8e] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-87882a8e

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test21fe6730fe484f3a92b1a16afe1bac8f-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scherchj5

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scherchj5

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5db6d9f47djrd27

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5db6d9f47djrd27

Unhealthy

Readiness probe failed: Get "https://10.134.0.49:8000/health": dial tcp 10.134.0.49:8000: connect: connection refused
(x10)

kserve-ci-e2e-test

OpenDataHubModelController

custom-route-timeout-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy custom-route-timeout-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/custom-route-timeout-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "custom-route-timeout-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/custom-route-timeout-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Service kserve-ci-e2e-test/custom-route-timeout-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Role kserve-ci-e2e-test/custom-route-timeout-test-epp-role

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-test-kserve

ScalingReplicaSet

Scaled up replica set custom-route-timeout-test-kserve-965ccd984 from 0 to 1

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set custom-route-timeout-test-kserve-router-scheduler-685c77879f from 0 to 1

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-test-kserve-router-scheduler-685c77879f

SuccessfulCreate

Created pod: custom-route-timeout-test-kserve-router-scheduler-685c7787ffkr6

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Service kserve-ci-e2e-test/custom-route-timeout-test-epp-service

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-test-kserve-965ccd984

SuccessfulCreate

Created pod: custom-route-timeout-test-kserve-965ccd984-nft7b

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

Created v1.Secret kserve-ci-e2e-test/custom-route-timeout-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-965ccd984-nft7b

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-685c7787ffkr6

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-965ccd984-nft7b

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-965ccd984-nft7b

Created

Created container: storage-initializer

kserve-ci-e2e-test

multus

custom-route-timeout-test-kserve-965ccd984-nft7b

AddedInterface

Add eth0 [10.134.0.51/23] from ovn-kubernetes

kserve-ci-e2e-test

multus

custom-route-timeout-test-kserve-router-scheduler-685c7787ffkr6

AddedInterface

Add eth0 [10.132.0.44/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-685c7787ffkr6

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-685c7787ffkr6

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-685c7787ffkr6

Created

Created container: main

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-685c7787ffkr6

Started

Started container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-685c7787ffkr6

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-965ccd984-nft7b

Created

Created container: main

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-965ccd984-nft7b

Started

Started container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-965ccd984-nft7b

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Updated

Updated v1.Secret kserve-ci-e2e-test/custom-route-timeout-test-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/custom-route-timeout-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/custom-route-timeout-test-kserve-route
(x11)

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-965ccd984-nft7b

Unhealthy

Startup probe failed: Get "https://10.134.0.51:8000/health": dial tcp 10.134.0.51:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

LLMInferenceServiceReady

LLMInferenceService [custom-route-timeout-test] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-685c7787ffkr6

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-router-scheduler-685c7787ffkr6

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

custom-route-timeout-test-kserve-965ccd984-nft7b

Killing

Stopping container main

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-799f46c59b

SuccessfulCreate

Created pod: router-gateway-1-openshift-default-799f46c59b-rbgv4
(x4)

kserve-ci-e2e-test

gateway_labeler_controller

router-gateway-1

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway router-gateway-1

kserve-ci-e2e-test

multus

router-gateway-1-openshift-default-799f46c59b-rbgv4

AddedInterface

Add eth0 [10.132.0.45/23] from ovn-kubernetes
(x4)

kserve-ci-e2e-test

deployment-controller

router-gateway-1-openshift-default

ScalingReplicaSet

Scaled up replica set router-gateway-1-openshift-default-799f46c59b from 0 to 1
(x2)

kserve-ci-e2e-test

deployment-controller

router-gateway-1-openshift-default

ScalingReplicaSet

Scaled down replica set router-gateway-1-openshift-default-54c47cdfc7 from 1 to 0

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-kgtbh

Killing

Stopping container istio-proxy

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-54c47cdfc7

SuccessfulDelete

Deleted pod: router-gateway-1-openshift-default-54c47cdfc7-kgtbh
(x3)

kserve-ci-e2e-test

deployment-controller

router-gateway-1-openshift-default

ScalingReplicaSet

Scaled up replica set router-gateway-1-openshift-default-54c47cdfc7 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-54c47cdfc7

SuccessfulCreate

Created pod: router-gateway-1-openshift-default-54c47cdfc7-r2bjb

kserve-ci-e2e-test

multus

router-gateway-1-openshift-default-54c47cdfc7-r2bjb

AddedInterface

Add eth0 [10.132.0.46/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-r2bjb

Unhealthy

Startup probe failed: Get "http://10.132.0.46:15021/healthz/ready": read tcp 10.132.0.2:35418->10.132.0.46:15021: read: connection reset by peer
(x8)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-r2bjb

Unhealthy

Startup probe failed: Get "http://10.132.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-rbgv4

Started

Started container istio-proxy
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-rbgv4

Created

Created container: istio-proxy
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-rbgv4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-r2bjb

Started

Started container istio-proxy
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-r2bjb

Created

Created container: istio-proxy
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-r2bjb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
(x6)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-rbgv4

Unhealthy

Startup probe failed: Get "http://10.132.0.45:15021/healthz/ready": dial tcp 10.132.0.45:15021: connect: connection refused

kserve-ci-e2e-test

replicaset-controller

router-with-refs-test-kserve-555479964d

SuccessfulCreate

Created pod: router-with-refs-test-kserve-555479964d-4kccs

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Service kserve-ci-e2e-test/router-with-refs-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Deployment kserve-ci-e2e-test/router-with-refs-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Secret kserve-ci-e2e-test/router-with-refs-test-kserve-self-signed-certs

kserve-ci-e2e-test

deployment-controller

router-with-refs-test-kserve

ScalingReplicaSet

Scaled up replica set router-with-refs-test-kserve-555479964d from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Deployment kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Service kserve-ci-e2e-test/router-with-refs-test-epp-service

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-555479964d-4kccs

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

replicaset-controller

router-with-refs-test-kserve-router-scheduler-7f865578d7

SuccessfulCreate

Created pod: router-with-refs-test-kserve-router-scheduler-7f865578d7-hrblf

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-555479964d-4kccs

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-555479964d-4kccs

Started

Started container storage-initializer
(x6)

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-54c47cdfc7-r2bjb

Unhealthy

Startup probe failed: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-7f865578d7-hrblf

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-7f865578d7-hrblf

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

multus

router-with-refs-test-kserve-router-scheduler-7f865578d7-hrblf

AddedInterface

Add eth0 [10.132.0.47/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.Role kserve-ci-e2e-test/router-with-refs-test-epp-role

kserve-ci-e2e-test

deployment-controller

router-with-refs-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set router-with-refs-test-kserve-router-scheduler-7f865578d7 from 0 to 1

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-7f865578d7-hrblf

Started

Started container storage-initializer

kserve-ci-e2e-test

multus

router-with-refs-test-kserve-555479964d-4kccs

AddedInterface

Add eth0 [10.134.0.52/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/router-with-refs-test-epp-rb
(x4)

kserve-ci-e2e-test

deployment-controller

router-gateway-1-openshift-default

ScalingReplicaSet

Scaled down replica set router-gateway-1-openshift-default-799f46c59b from 1 to 0

kserve-ci-e2e-test

replicaset-controller

router-gateway-1-openshift-default-799f46c59b

SuccessfulDelete

Deleted pod: router-gateway-1-openshift-default-799f46c59b-rbgv4
(x5)

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/router-with-refs-test-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-7f865578d7-hrblf

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-7f865578d7-hrblf

Created

Created container: main

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-7f865578d7-hrblf

Started

Started container main

kserve-ci-e2e-test

kubelet

router-gateway-1-openshift-default-799f46c59b-rbgv4

Killing

Stopping container istio-proxy

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-555479964d-4kccs

Started

Started container main

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-555479964d-4kccs

Created

Created container: main

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-555479964d-4kccs

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Updated

Updated v1.Secret kserve-ci-e2e-test/router-with-refs-test-kserve-self-signed-certs
(x11)

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-555479964d-4kccs

Unhealthy

Startup probe failed: Get "https://10.134.0.52:8000/health": dial tcp 10.134.0.52:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

LLMInferenceServiceReady

LLMInferenceService [router-with-refs-test] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-555479964d-4kccs

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-7f865578d7-hrblf

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-router-scheduler-7f865578d7-hrblf

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

router-with-refs-test-kserve-555479964d-4kccs

Unhealthy

Readiness probe failed: Get "https://10.134.0.52:8000/health": dial tcp 10.134.0.52:8000: connect: connection refused
(x12)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-50bc673d

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-50bc673d-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-50bc673d-kserve-route" not found

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-50bc673d-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-50bc673d-kserve-84599bcd6b from 0 to 1

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-84599bcd6b

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-50bc673d-kserve-84599bcd6b4vvr6

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test00d7278d8a22c4e39146a6b0eb840f45-llmisvc-mode

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

Created v1.Secret kserve-ci-e2e-test/llmisv44d181485fad85e662eb092f3749502f-kserve-self-signed-certs

kserve-ci-e2e-test

deployment-controller

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheduler-69d8c68fd8 from 0 to 1

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-6f89477b49 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-6f89477b49

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-6f654d9

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-6f654d9

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-6f654d9

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-6f654d9

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-6f654d9

AddedInterface

Add eth0 [10.134.0.54/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheduler-69d8c68fd8

SuccessfulCreate

Created pod: llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schenm4bp

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-84599bcd6b4vvr6

Pulling

Pulling image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2"

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-84599bcd6b4vvr6

AddedInterface

Add eth0 [10.134.0.53/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-84599bcd6b4vvr6

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" in 778ms (778ms including waiting). Image size: 44914394 bytes.

kserve-ci-e2e-test

kubelet

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schenm4bp

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schenm4bp

Created

Created container: storage-initializer

kserve-ci-e2e-test

multus

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schenm4bp

AddedInterface

Add eth0 [10.132.0.48/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-84599bcd6b4vvr6

Created

Created container: llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-84599bcd6b4vvr6

Started

Started container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schenm4bp

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schenm4bp

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-84599bcd6b4vvr6

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-84599bcd6b4vvr6

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schenm4bp

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-84599bcd6b4vvr6

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schenm4bp

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-6f654d9

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-6f654d9

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-6f654d9

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-84599bcd6b4vvr6

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-84599bcd6b4vvr6

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-84599bcd6b4vvr6

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv44d181485fad85e662eb092f3749502f-kserve-self-signed-certs
(x10)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-shadow-svc
(x13)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-84599bcd6b4vvr6

Unhealthy

Startup probe failed: Get "https://10.134.0.53:8001/health": dial tcp 10.134.0.53:8001: connect: connection refused
(x15)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-6f654d9

Unhealthy

Startup probe failed: Get "https://10.134.0.54:8000/health": dial tcp 10.134.0.54:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-fb-opt-125m-route-50bc673d] is Ready

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-6f654d9

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-50bc673d

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test00d7278d8a22c4e39146a6b0eb840f45-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schenm4bp

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schenm4bp

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-84599bcd6b4vvr6

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-50bc673d-kserve-84599bcd6b4vvr6

Killing

Stopping container llm-d-routing-sidecar
(x13)

kserve-ci-e2e-test

OpenDataHubModelController

custom-route-timeout-pd-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy custom-route-timeout-pd-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "custom-route-timeout-pd-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-pd-test-kserve

ScalingReplicaSet

Scaled up replica set custom-route-timeout-pd-test-kserve-6d9d59446f from 0 to 1

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-pd-test-kserve-6d9d59446f

SuccessfulCreate

Created pod: custom-route-timeout-pd-test-kserve-6d9d59446f-spptn

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-pd-test-kserve-prefill-f7f58c4f6

SuccessfulCreate

Created pod: custom-route-timeout-pd-test-kserve-prefill-f7f58c4f6-nsctq

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Service kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-workload-svc

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-pd-test-kserve-prefill

ScalingReplicaSet

Scaled up replica set custom-route-timeout-pd-test-kserve-prefill-f7f58c4f6 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-rb

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Role kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-role

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.Secret kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-f7f58c4f6-nsctq

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-f7f58c4f6-nsctq

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

deployment-controller

custom-route-timeout-pd-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set custom-route-timeout-pd-test-kserve-router-scheduler-5f5fcc46c8 from 0 to 1

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-6d9d59446f-spptn

Started

Started container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-6d9d59446f-spptn

Created

Created container: llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-6d9d59446f-spptn

Pulled

Container image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

multus

custom-route-timeout-pd-test-kserve-6d9d59446f-spptn

AddedInterface

Add eth0 [10.134.0.55/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-pd-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-5f5fcfmn6z

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-5f5fcfmn6z

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-5f5fcfmn6z

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

multus

custom-route-timeout-pd-test-kserve-router-scheduler-5f5fcfmn6z

AddedInterface

Add eth0 [10.132.0.49/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

custom-route-timeout-pd-test-kserve-router-scheduler-5f5fcc46c8

SuccessfulCreate

Created pod: custom-route-timeout-pd-test-kserve-router-scheduler-5f5fcfmn6z

kserve-ci-e2e-test

multus

custom-route-timeout-pd-test-kserve-prefill-f7f58c4f6-nsctq

AddedInterface

Add eth0 [10.134.0.56/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-pd-test-epp-sa

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-f7f58c4f6-nsctq

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-6d9d59446f-spptn

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-6d9d59446f-spptn

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-6d9d59446f-spptn

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-5f5fcfmn6z

Created

Created container: main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-5f5fcfmn6z

Started

Started container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-5f5fcfmn6z

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-f7f58c4f6-nsctq

Started

Started container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-f7f58c4f6-nsctq

Created

Created container: main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-f7f58c4f6-nsctq

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-6d9d59446f-spptn

Created

Created container: main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-6d9d59446f-spptn

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-6d9d59446f-spptn

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Updated

Updated v1.Secret kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-route
(x10)

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-shadow-svc
(x15)

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-6d9d59446f-spptn

Unhealthy

Startup probe failed: Get "https://10.134.0.55:8001/health": dial tcp 10.134.0.55:8001: connect: connection refused
(x15)

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-f7f58c4f6-nsctq

Unhealthy

Startup probe failed: Get "https://10.134.0.56:8000/health": dial tcp 10.134.0.56:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

LLMInferenceServiceReady

LLMInferenceService [custom-route-timeout-pd-test] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

custom-route-timeout-pd-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-pd-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-6d9d59446f-spptn

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-prefill-f7f58c4f6-nsctq

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-5f5fcfmn6z

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-router-scheduler-5f5fcfmn6z

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

custom-route-timeout-pd-test-kserve-6d9d59446f-spptn

Killing

Stopping container llm-d-routing-sidecar
(x13)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-pvc-router-manage-e8706282

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-pvc-router-manage-e8706282-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-pvc-router-manage-e8706282-kserve-route" not found

kserve-ci-e2e-test

replicaset-controller

router-gateway-2-openshift-default-54c789bdc6

SuccessfulCreate

Created pod: router-gateway-2-openshift-default-54c789bdc6-chk28

kserve-ci-e2e-test

gateway_labeler_controller

router-gateway-2

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway router-gateway-2

kserve-ci-e2e-test

service-controller

router-gateway-2-openshift-default

EnsuringLoadBalancer

Ensuring load balancer

kserve-ci-e2e-test

deployment-controller

router-gateway-2-openshift-default

ScalingReplicaSet

Scaled up replica set router-gateway-2-openshift-default-54c789bdc6 from 0 to 1

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-54c789bdc6-chk28

Created

Created container: istio-proxy

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-54c789bdc6-chk28

Started

Started container istio-proxy

kserve-ci-e2e-test

multus

router-gateway-2-openshift-default-54c789bdc6-chk28

AddedInterface

Add eth0 [10.132.0.50/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-54c789bdc6-chk28

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine

kserve-ci-e2e-test

deployment-controller

router-gateway-2-openshift-default

ScalingReplicaSet

Scaled up replica set router-gateway-2-openshift-default-684dc9b4b9 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

router-gateway-2-openshift-default-684dc9b4b9

SuccessfulCreate

Created pod: router-gateway-2-openshift-default-684dc9b4b9-c9txx

kserve-ci-e2e-test

deployment-controller

llmisvc-model-pvc-router-manage-e8706282-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.Secret kserve-ci-e2e-test/llmisvb19f98874e050eec8ca94d49676113f0-kserve-self-signed-certs

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c

SuccessfulCreate

Created pod: llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-mhsdz

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-rb

kserve-ci-e2e-test

multus

router-gateway-2-openshift-default-684dc9b4b9-c9txx

AddedInterface

Add eth0 [10.132.0.51/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-workload-svc

kserve-ci-e2e-test

deployment-controller

llmisvc-model-pvc-router-manage-e8706282-kserve-prefill

ScalingReplicaSet

Scaled up replica set llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79497db4cc from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-epp-sa

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79497db4cc

SuccessfulCreate

Created pod: llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79j8gwt

kserve-ci-e2e-test

service-controller

router-gateway-2-openshift-default

EnsuredLoadBalancer

Ensured load balancer

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-prefill

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve

kserve-ci-e2e-test

multus

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehlrzb

AddedInterface

Add eth0 [10.132.0.52/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheduler-67b4bb9646

SuccessfulCreate

Created pod: llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehlrzb

kserve-ci-e2e-test

kubelet

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehlrzb

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehlrzb

Started

Started container main

kserve-ci-e2e-test

attachdetach-controller

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehlrzb

FailedAttachVolume

Multi-Attach error for volume "pvc-3823dadc-7c52-412c-8ef9-84caab00a904" Volume is already used by pod(s) llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemgcff

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-54c789bdc6-chk28

Killing

Stopping container istio-proxy

kserve-ci-e2e-test

deployment-controller

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheduler-67b4bb9646 from 0 to 1

kserve-ci-e2e-test

kubelet

llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehlrzb

Created

Created container: main

kserve-ci-e2e-test

replicaset-controller

router-gateway-2-openshift-default-54c789bdc6

SuccessfulDelete

Deleted pod: router-gateway-2-openshift-default-54c789bdc6-chk28

kserve-ci-e2e-test

deployment-controller

router-gateway-2-openshift-default

ScalingReplicaSet

Scaled down replica set router-gateway-2-openshift-default-54c789bdc6 from 1 to 0

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-testf5d060a5eb39a04e074b78907a1556a6-llmisvc-mode
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-684dc9b4b9-c9txx

Started

Started container istio-proxy
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-684dc9b4b9-c9txx

Created

Created container: istio-proxy
(x2)

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-684dc9b4b9-c9txx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
(x5)

kserve-ci-e2e-test

kubelet

router-gateway-2-openshift-default-684dc9b4b9-c9txx

Unhealthy

Startup probe failed: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Deployment kserve-ci-e2e-test/router-with-refs-pd-test-kserve

kserve-ci-e2e-test

multus

router-with-refs-pd-test-kserve-664bd66db5-cjjg6

AddedInterface

Add eth0 [10.134.0.59/23] from ovn-kubernetes

kserve-ci-e2e-test

multus

router-with-refs-pd-test-kserve-prefill-9d68b57d4-trssd

AddedInterface

Add eth0 [10.134.0.60/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-9d68b57d4-trssd

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-9d68b57d4-trssd

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-9d68b57d4-trssd

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-664bd66db5-cjjg6

Started

Started container llm-d-routing-sidecar

kserve-ci-e2e-test

replicaset-controller

router-with-refs-pd-test-kserve-prefill-9d68b57d4

SuccessfulCreate

Created pod: router-with-refs-pd-test-kserve-prefill-9d68b57d4-trssd

kserve-ci-e2e-test

deployment-controller

router-with-refs-pd-test-kserve-prefill

ScalingReplicaSet

Scaled up replica set router-with-refs-pd-test-kserve-prefill-9d68b57d4 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

router-with-refs-pd-test-kserve-router-scheduler-5c75cd7b8b

SuccessfulCreate

Created pod: router-with-refs-pd-test-kserve-router-scheduler-5c75cd7b8gcvgz

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-664bd66db5-cjjg6

Created

Created container: llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-664bd66db5-cjjg6

Pulled

Container image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-pd-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-pd-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Service kserve-ci-e2e-test/router-with-refs-pd-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Deployment kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill

kserve-ci-e2e-test

replicaset-controller

router-with-refs-pd-test-kserve-664bd66db5

SuccessfulCreate

Created pod: router-with-refs-pd-test-kserve-664bd66db5-cjjg6

kserve-ci-e2e-test

deployment-controller

router-with-refs-pd-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set router-with-refs-pd-test-kserve-router-scheduler-5c75cd7b8b from 0 to 1

kserve-ci-e2e-test

deployment-controller

router-with-refs-pd-test-kserve

ScalingReplicaSet

Scaled up replica set router-with-refs-pd-test-kserve-664bd66db5 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Secret kserve-ci-e2e-test/router-with-refs-pd-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-pd-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.Role kserve-ci-e2e-test/router-with-refs-pd-test-kserve-role

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/router-with-refs-pd-test-kserve-rb

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-664bd66db5-cjjg6

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-5c75cd7b8gcvgz

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

multus

router-with-refs-pd-test-kserve-router-scheduler-5c75cd7b8gcvgz

AddedInterface

Add eth0 [10.132.0.53/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-5c75cd7b8gcvgz

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-5c75cd7b8gcvgz

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-664bd66db5-cjjg6

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-664bd66db5-cjjg6

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-5c75cd7b8gcvgz

Created

Created container: main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-5c75cd7b8gcvgz

Started

Started container main
(x9)

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/router-with-refs-pd-test-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-5c75cd7b8gcvgz

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-9d68b57d4-trssd

Created

Created container: main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-9d68b57d4-trssd

Started

Started container main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-9d68b57d4-trssd

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-664bd66db5-cjjg6

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-664bd66db5-cjjg6

Created

Created container: main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-664bd66db5-cjjg6

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-route
(x10)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Updated

Updated v1.Secret kserve-ci-e2e-test/router-with-refs-pd-test-kserve-self-signed-certs
(x16)

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-9d68b57d4-trssd

Unhealthy

Startup probe failed: Get "https://10.134.0.60:8000/health": dial tcp 10.134.0.60:8000: connect: connection refused
(x16)

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-664bd66db5-cjjg6

Unhealthy

Startup probe failed: Get "https://10.134.0.59:8001/health": dial tcp 10.134.0.59:8001: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

LLMInferenceServiceReady

LLMInferenceService [router-with-refs-pd-test] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

router-with-refs-pd-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-pd-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-664bd66db5-cjjg6

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-664bd66db5-cjjg6

Killing

Stopping container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-5c75cd7b8gcvgz

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-router-scheduler-5c75cd7b8gcvgz

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

router-with-refs-pd-test-kserve-prefill-9d68b57d4-trssd

Killing

Stopping container main
(x14)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-3c960099

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-3c960099-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-3c960099-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-3c960099

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-fb-opt-125m-route-3c960099-kserve-5c94bf7bb7

SuccessfulCreate

Created pod: llmisvc-model-fb-opt-125m-route-3c960099-kserve-5c94bf7bb75w9xs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-3c960099

Created

Created v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-3c960099

Created

Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-workload-svc

kserve-ci-e2e-test

deployment-controller

llmisvc-model-fb-opt-125m-route-3c960099-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-fb-opt-125m-route-3c960099-kserve-5c94bf7bb7 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-3c960099

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-3c960099

Created

Created v1.Secret kserve-ci-e2e-test/llmisvd10b93c8eba12f87c3a350a5cae2ee0c-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-5c94bf7bb75w9xs

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-3c960099-kserve-5c94bf7bb75w9xs

AddedInterface

Add eth0 [10.134.0.61/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-5c94bf7bb75w9xs

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-5c94bf7bb75w9xs

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-5c94bf7bb75w9xs

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-5c94bf7bb75w9xs

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-5c94bf7bb75w9xs

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-3c960099

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisvd10b93c8eba12f87c3a350a5cae2ee0c-kserve-self-signed-certs
(x11)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-5c94bf7bb75w9xs

Unhealthy

Startup probe failed: Get "https://10.134.0.61:8000/health": dial tcp 10.134.0.61:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-3c960099

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-fb-opt-125m-route-3c960099] is Ready

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-5c94bf7bb75w9xs

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-3c960099-kserve-5c94bf7bb75w9xs

Unhealthy

Readiness probe failed: Get "https://10.134.0.61:8000/health": dial tcp 10.134.0.61:8000: connect: connection refused
(x13)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-fb-opt-125m-route-dc21cb14

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-route" not found

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

AddedInterface

Add eth0 [10.134.0.62/23] from ovn-kubernetes

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

GroupsProgressing

Created worker statefulset for leader pod llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.Secret kserve-ci-e2e-test/llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

CreatingRevision

Creating revision with key 848c75bc96 for a newly created LeaderWorkerSet

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-epp-role

kserve-ci-e2e-test

replicaset-controller

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-scheduler-6b4848c4bd

SuccessfulCreate

Created pod: llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schefs4wn

kserve-ci-e2e-test

deployment-controller

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-scheduler-6b4848c4bd from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-epp-sa

kserve-ci-e2e-test

statefulset-controller

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

SuccessfulCreate

create Pod llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1 in StatefulSet llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0 successful

kserve-ci-e2e-test

multus

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

AddedInterface

Add eth0 [10.132.0.55/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-workload-svc

kserve-ci-e2e-test

multus

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schefs4wn

AddedInterface

Add eth0 [10.132.0.54/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-scc

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schefs4wn

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.LeaderWorkerSet kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

kserve-ci-e2e-test

statefulset-controller

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

SuccessfulCreate

create Pod llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0 in StatefulSet llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn successful
(x2)

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

GroupsProgressing

Replicas are progressing, with 0 groups ready of total 1 groups

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test386bd5808c5c450f11fd8632e1f821fb-llmisvc-mode

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

GroupsProgressing

Created leader statefulset llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schefs4wn

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schefs4wn

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schefs4wn

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schefs4wn

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schefs4wn

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Pulling

Pulling image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0"

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-route
(x8)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Pulled

Successfully pulled image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" in 27.76s (27.76s including waiting). Image size: 3531177328 bytes.

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Created

Created container: main
(x9)

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Unhealthy

Startup probe failed: Get "https://10.134.0.62:8000/health": dial tcp 10.134.0.62:8000: connect: connection refused
(x2)

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn

AllGroupsReady

All replicas are ready, with 1 groups ready of total 1 groups
(x14)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-pvc-router-manage-2577e794

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-pvc-router-manage-2577e794-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-pvc-router-manage-2577e794-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.LeaderWorkerSet kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.Secret kserve-ci-e2e-test/llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs

kserve-ci-e2e-test

statefulset-controller

llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

SuccessfulCreate

create Pod llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1 in StatefulSet llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0 successful

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test37591b20e96e9663d45a730d03070f1e-llmisvc-mode

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-epp-rb

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-pvc-router-manage-2577e794-kserve-mn

CreatingRevision

Creating revision with key 686d468674 for a newly created LeaderWorkerSet

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-pvc-router-manage-2577e794-kserve-mn

GroupsProgressing

Created leader statefulset llmisvc-model-pvc-router-manage-2577e794-kserve-mn

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-scc

kserve-ci-e2e-test

deployment-controller

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheduler-7dbcb75dbc from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-workload-svc

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-pvc-router-manage-2577e794-kserve-mn

GroupsProgressing

Created worker statefulset for leader pod llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0

kserve-ci-e2e-test

statefulset-controller

llmisvc-model-pvc-router-manage-2577e794-kserve-mn

SuccessfulCreate

create Pod llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0 in StatefulSet llmisvc-model-pvc-router-manage-2577e794-kserve-mn successful
(x2)

kserve-ci-e2e-test

leaderworkerset

llmisvc-model-pvc-router-manage-2577e794-kserve-mn

GroupsProgressing

Replicas are progressing, with 0 groups ready of total 1 groups

kserve-ci-e2e-test

replicaset-controller

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheduler-7dbcb75dbc

SuccessfulCreate

Created pod: llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-schelqsd2

kserve-ci-e2e-test

kubelet

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-schelqsd2

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-schelqsd2

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-schelqsd2

Created

Created container: main

kserve-ci-e2e-test

multus

llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-schelqsd2

AddedInterface

Add eth0 [10.133.0.42/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-fb-opt-125m-route-dc21cb14] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-route
(x8)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-shadow-svc

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schefs4wn

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-fb-opt-125m-route-dc21cb14

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test386bd5808c5c450f11fd8632e1f821fb-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schefs4wn

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0

Unhealthy

Readiness probe failed: Get "https://10.134.0.62:8000/health": dial tcp 10.134.0.62:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1

Killing

Stopping container main
(x14)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-inline-config-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-inline-config-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-inline-config-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-inline-config-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-inline-config-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-inline-config-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-inline-config-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-inline-config-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-inline-config-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-inline-config-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-inline-config-test-kserve

kserve-ci-e2e-test

replicaset-controller

scheduler-inline-config-test-kserve-router-scheduler-d9ddf6c4d

SuccessfulCreate

Created pod: scheduler-inline-config-test-kserve-router-scheduler-d9ddfrbp26

kserve-ci-e2e-test

deployment-controller

scheduler-inline-config-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-inline-config-test-kserve-router-scheduler-d9ddf6c4d from 0 to 1

kserve-ci-e2e-test

deployment-controller

scheduler-inline-config-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-inline-config-test-kserve-7c6c58cc96 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

scheduler-inline-config-test-kserve-7c6c58cc96

SuccessfulCreate

Created pod: scheduler-inline-config-test-kserve-7c6c58cc96-t4d7s

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-inline-config-test-kserve-self-signed-certs

kserve-ci-e2e-test

multus

scheduler-inline-config-test-kserve-router-scheduler-d9ddfrbp26

AddedInterface

Add eth0 [10.132.0.57/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-d9ddfrbp26

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-d9ddfrbp26

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-7c6c58cc96-t4d7s

Started

Started container main

kserve-ci-e2e-test

multus

scheduler-inline-config-test-kserve-7c6c58cc96-t4d7s

AddedInterface

Add eth0 [10.133.0.43/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-7c6c58cc96-t4d7s

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-d9ddfrbp26

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-7c6c58cc96-t4d7s

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-inline-config-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-inline-config-test-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-inline-config-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

LLMInferenceServiceReady

LLMInferenceService [scheduler-inline-config-test] is Ready

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-7c6c58cc96-t4d7s

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-inline-config-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-inline-config-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-d9ddfrbp26

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

scheduler-inline-config-test-kserve-router-scheduler-d9ddfrbp26

Killing

Stopping container main
(x14)

kserve-ci-e2e-test

OpenDataHubModelController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4hjlzp

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.Deployment kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.Role kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test80f99418f0f3eb4c80f02e6144a6d4f2-llmisvc-mode

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

Created v1.Secret kserve-ci-e2e-test/llmisv431d7e30f13554296364403a5b6365f5-kserve-self-signed-certs

kserve-ci-e2e-test

deployment-controller

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve

ScalingReplicaSet

Scaled up replica set llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scheduler-79866dc455

SuccessfulCreate

Created pod: llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche6fld4

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4hjlzp

Created

Created container: main

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4hjlzp

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

multus

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4hjlzp

AddedInterface

Add eth0 [10.133.0.44/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4

SuccessfulCreate

Created pod: llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4hjlzp

kserve-ci-e2e-test

deployment-controller

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scheduler-79866dc455 from 0 to 1

kserve-ci-e2e-test

kubelet

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche6fld4

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche6fld4

Started

Started container main

kserve-ci-e2e-test

kubelet

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche6fld4

Created

Created container: main

kserve-ci-e2e-test

multus

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche6fld4

AddedInterface

Add eth0 [10.132.0.58/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv431d7e30f13554296364403a5b6365f5-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

LLMInferenceServiceReady

LLMInferenceService [llmisvc-model-qwen2-5-0-5b-rout-4c1ba212] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test80f99418f0f3eb4c80f02e6144a6d4f2-llmisvc-mode

kserve-ci-e2e-test

kubelet

llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4hjlzp

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche6fld4

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche6fld4

FailedPreStopHook

PreStopHook failed
(x15)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-configmap-ref-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-configmap-ref-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-configmap-ref-test-kserve-route" not found

kserve-ci-e2e-test

replicaset-controller

scheduler-configmap-ref-test-kserve-router-scheduler-68f876cc5d

SuccessfulCreate

Created pod: scheduler-configmap-ref-test-kserve-router-scheduler-68f87bprxs

kserve-ci-e2e-test

replicaset-controller

scheduler-configmap-ref-test-kserve-744bf59fbd

SuccessfulCreate

Created pod: scheduler-configmap-ref-test-kserve-744bf59fbd-chn2n

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-744bf59fbd-chn2n

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-744bf59fbd-chn2n

Started

Started container main

kserve-ci-e2e-test

multus

scheduler-configmap-ref-test-kserve-744bf59fbd-chn2n

AddedInterface

Add eth0 [10.133.0.45/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-service

kserve-ci-e2e-test

deployment-controller

scheduler-configmap-ref-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-configmap-ref-test-kserve-744bf59fbd from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-configmap-ref-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-744bf59fbd-chn2n

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

deployment-controller

scheduler-configmap-ref-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-configmap-ref-test-kserve-router-scheduler-68f876cc5d from 0 to 1

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-68f87bprxs

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-68f87bprxs

Created

Created container: main

kserve-ci-e2e-test

multus

scheduler-configmap-ref-test-kserve-router-scheduler-68f87bprxs

AddedInterface

Add eth0 [10.132.0.59/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-68f87bprxs

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

LLMInferenceServiceReady

LLMInferenceService [scheduler-configmap-ref-test] is Ready

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-68f87bprxs

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-configmap-ref-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-configmap-ref-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-router-scheduler-68f87bprxs

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-744bf59fbd-chn2n

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-configmap-ref-test-kserve-744bf59fbd-chn2n

Unhealthy

Readiness probe failed: Get "https://10.133.0.45:8000/health": dial tcp 10.133.0.45:8000: connect: connection refused
(x15)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-ha-replicas-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-ha-replicas-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-ha-replicas-test-kserve-route" not found

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dbch5s

Created

Created container: main

kserve-ci-e2e-test

deployment-controller

scheduler-ha-replicas-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-ha-replicas-test-kserve-5c5cfcf776 from 0 to 1

kserve-ci-e2e-test

multus

scheduler-ha-replicas-test-kserve-5c5cfcf776-9djm5

AddedInterface

Add eth0 [10.133.0.46/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-5c5cfcf776-9djm5

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-5c5cfcf776-9djm5

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-5c5cfcf776-9djm5

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-ha-replicas-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs

kserve-ci-e2e-test

replicaset-controller

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dbc4

SuccessfulCreate

Created pod: scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dbch5s

kserve-ci-e2e-test

deployment-controller

scheduler-ha-replicas-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dbc4 from 0 to 2

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d9qw9w

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d9qw9w

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dbch5s

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dbch5s

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

multus

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dbch5s

AddedInterface

Add eth0 [10.132.0.60/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dbc4

SuccessfulCreate

Created pod: scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d9qw9w

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d9qw9w

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

multus

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d9qw9w

AddedInterface

Add eth0 [10.133.0.47/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

scheduler-ha-replicas-test-kserve-5c5cfcf776

SuccessfulCreate

Created pod: scheduler-ha-replicas-test-kserve-5c5cfcf776-9djm5

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

LLMInferenceServiceReady

LLMInferenceService [scheduler-ha-replicas-test] is Ready

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dbch5s

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d9qw9w

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-ha-replicas-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-ha-replicas-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dbch5s

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d9qw9w

Killing

Stopping container main
(x8)

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d9qw9w

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

kserve-ci-e2e-test

kubelet

scheduler-ha-replicas-test-kserve-5c5cfcf776-9djm5

Killing

Stopping container main
(x13)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-custom-template-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-custom-template-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-custom-template-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-custom-template-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-custom-template-test-kserve-self-signed-certs

kserve-ci-e2e-test

replicaset-controller

scheduler-custom-template-test-kserve-5bc7bd6864

SuccessfulCreate

Created pod: scheduler-custom-template-test-kserve-5bc7bd6864-28w56

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-custom-template-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-custom-template-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-custom-template-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-custom-template-test-epp-role

kserve-ci-e2e-test

deployment-controller

scheduler-custom-template-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-custom-template-test-kserve-5bc7bd6864 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-custom-template-test-kserve

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-5bc7bd6864-28w56

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-custom-template-test-epp-rb

kserve-ci-e2e-test

multus

scheduler-custom-template-test-kserve-5bc7bd6864-28w56

AddedInterface

Add eth0 [10.133.0.48/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-router-scheduler-567bmx62

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-router-scheduler-567bmx62

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-router-scheduler-567bmx62

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

replicaset-controller

scheduler-custom-template-test-kserve-router-scheduler-567469f665

SuccessfulCreate

Created pod: scheduler-custom-template-test-kserve-router-scheduler-567bmx62

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-5bc7bd6864-28w56

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-5bc7bd6864-28w56

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-custom-template-test-epp-service

kserve-ci-e2e-test

multus

scheduler-custom-template-test-kserve-router-scheduler-567bmx62

AddedInterface

Add eth0 [10.132.0.61/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

scheduler-custom-template-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-custom-template-test-kserve-router-scheduler-567469f665 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-custom-template-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-custom-template-test-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-custom-template-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

LLMInferenceServiceReady

LLMInferenceService [scheduler-custom-template-test] is Ready

kserve-ci-e2e-test

deployment-controller

stop-feature-test-kserve

ScalingReplicaSet

Scaled up replica set stop-feature-test-kserve-5f875cbf68 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve

kserve-ci-e2e-test

replicaset-controller

stop-feature-test-kserve-5f875cbf68

SuccessfulCreate

Created pod: stop-feature-test-kserve-5f875cbf68-7df7h

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Service kserve-ci-e2e-test/stop-feature-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/stop-feature-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-stop-feature-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Role kserve-ci-e2e-test/stop-feature-test-epp-role

kserve-ci-e2e-test

deployment-controller

stop-feature-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set stop-feature-test-kserve-router-scheduler-76cf8bfb8f from 0 to 1

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-5f875cbf68-7df7h

Started

Started container storage-initializer

kserve-ci-e2e-test

multus

stop-feature-test-kserve-router-scheduler-76cf8bfb8f-75mhj

AddedInterface

Add eth0 [10.132.0.63/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-76cf8bfb8f-75mhj

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-76cf8bfb8f-75mhj

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-76cf8bfb8f-75mhj

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-5f875cbf68-7df7h

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-5f875cbf68-7df7h

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

multus

stop-feature-test-kserve-5f875cbf68-7df7h

AddedInterface

Add eth0 [10.132.0.62/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

stop-feature-test-kserve-router-scheduler-76cf8bfb8f

SuccessfulCreate

Created pod: stop-feature-test-kserve-router-scheduler-76cf8bfb8f-75mhj

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/stop-feature-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

Created v1.Service kserve-ci-e2e-test/stop-feature-test-epp-service

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-76cf8bfb8f-75mhj

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-76cf8bfb8f-75mhj

Created

Created container: main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-76cf8bfb8f-75mhj

Started

Started container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-5f875cbf68-7df7h

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-5f875cbf68-7df7h

Started

Started container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-5f875cbf68-7df7h

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-custom-template-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-custom-template-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-router-scheduler-567bmx62

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-router-scheduler-567bmx62

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-custom-template-test-kserve-5bc7bd6864-28w56

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/stop-feature-test-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/stop-feature-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Updated

Updated v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs
(x15)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-v06-pd-migration-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-v06-pd-migration-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-v06-pd-migration-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill

kserve-ci-e2e-test

deployment-controller

scheduler-v06-pd-migration-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-v06-pd-migration-test-kserve-5c87756974 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-self-signed-certs

kserve-ci-e2e-test

replicaset-controller

scheduler-v06-pd-migration-test-kserve-5c87756974

SuccessfulCreate

Created pod: scheduler-v06-pd-migration-test-kserve-5c87756974-hnm4k

kserve-ci-e2e-test

deployment-controller

scheduler-v06-pd-migration-test-kserve-prefill

ScalingReplicaSet

Scaled up replica set scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-role

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-router-scheduler-59pfml2

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-6ppgx

Started

Started container main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-pd-migration-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-v06-pd-migration-test-epp-auth-rb

kserve-ci-e2e-test

multus

scheduler-v06-pd-migration-test-kserve-5c87756974-hnm4k

AddedInterface

Add eth0 [10.133.0.49/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

scheduler-v06-pd-migration-test-kserve-router-scheduler-59dc679cd

SuccessfulCreate

Created pod: scheduler-v06-pd-migration-test-kserve-router-scheduler-59pfml2

kserve-ci-e2e-test

deployment-controller

scheduler-v06-pd-migration-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-v06-pd-migration-test-kserve-router-scheduler-59dc679cd from 0 to 1

kserve-ci-e2e-test

multus

scheduler-v06-pd-migration-test-kserve-router-scheduler-59pfml2

AddedInterface

Add eth0 [10.133.0.51/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-6ppgx

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-router-scheduler-59pfml2

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-6ppgx

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-router-scheduler-59pfml2

Started

Started container main

kserve-ci-e2e-test

multus

scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-6ppgx

AddedInterface

Add eth0 [10.133.0.50/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-workload-svc

kserve-ci-e2e-test

replicaset-controller

scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847

SuccessfulCreate

Created pod: scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-6ppgx

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-hnm4k

Pulling

Pulling image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2"

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-hnm4k

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-hnm4k

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" in 831ms (831ms including waiting). Image size: 44914394 bytes.

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-hnm4k

Started

Started container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-hnm4k

Created

Created container: llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-hnm4k

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-hnm4k

Created

Created container: main
(x11)

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-5f875cbf68-7df7h

Unhealthy

Startup probe failed: Get "https://10.132.0.62:8000/health": dial tcp 10.132.0.62:8000: connect: connection refused
(x10)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

LLMInferenceServiceReady

LLMInferenceService [scheduler-v06-pd-migration-test] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-76cf8bfb8f-75mhj

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Service kserve-ci-e2e-test/stop-feature-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.ServiceAccount kserve-ci-e2e-test/stop-feature-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.Service kserve-ci-e2e-test/stop-feature-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

Deleted

Deleted v1.InferencePool kserve-ci-e2e-test/stop-feature-test-inference-pool

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-5f875cbf68-7df7h

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-76cf8bfb8f-75mhj

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

LLMInferenceServiceNotReady

LLMInferenceService [stop-feature-test] is no longer Ready because of: GatewaysReady, HTTPRoutesReady, InferencePoolReady, MainWorkloadReady, PrefillWorkerWorkloadReady, PrefillWorkloadReady, RouterReady, SchedulerWorkloadReady, WorkerWorkloadReady, WorkloadsReady
(x25)

kserve-ci-e2e-test

OpenDataHubModelController

stop-feature-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy stop-feature-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/stop-feature-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "stop-feature-test-kserve-route" not found

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-hnm4k

Killing

Stopping container llm-d-routing-sidecar

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-pd-migration-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-v06-pd-migration-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-hnm4k

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-router-scheduler-59pfml2

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-router-scheduler-59pfml2

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-6ppgx

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-5c87756974-hnm4k

Unhealthy

Readiness probe failed: Get "https://10.133.0.49:8001/health": dial tcp 10.133.0.49:8001: connect: connection refused

kserve-ci-e2e-test

kubelet

scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-6ppgx

Unhealthy

Readiness probe failed: Get "https://10.133.0.50:8000/health": dial tcp 10.133.0.50:8000: connect: connection refused

kserve-ci-e2e-test

replicaset-controller

stop-feature-test-kserve-router-scheduler-76cf8bfb8f

SuccessfulCreate

Created pod: stop-feature-test-kserve-router-scheduler-76cf8bfb8f-qgsf9

kserve-ci-e2e-test

deployment-controller

stop-feature-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set stop-feature-test-kserve-router-scheduler-76cf8bfb8f from 0 to 1

kserve-ci-e2e-test

deployment-controller

stop-feature-test-kserve

ScalingReplicaSet

Scaled up replica set stop-feature-test-kserve-5f875cbf68 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

stop-feature-test-kserve-5f875cbf68

SuccessfulCreate

Created pod: stop-feature-test-kserve-5f875cbf68-x2lqh

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-5f875cbf68-x2lqh

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

multus

stop-feature-test-kserve-router-scheduler-76cf8bfb8f-qgsf9

AddedInterface

Add eth0 [10.132.0.65/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-76cf8bfb8f-qgsf9

Started

Started container storage-initializer

kserve-ci-e2e-test

multus

stop-feature-test-kserve-5f875cbf68-x2lqh

AddedInterface

Add eth0 [10.132.0.64/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-76cf8bfb8f-qgsf9

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-76cf8bfb8f-qgsf9

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-5f875cbf68-x2lqh

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-5f875cbf68-x2lqh

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-76cf8bfb8f-qgsf9

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-76cf8bfb8f-qgsf9

Started

Started container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-76cf8bfb8f-qgsf9

Created

Created container: main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-5f875cbf68-x2lqh

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-5f875cbf68-x2lqh

Started

Started container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-5f875cbf68-x2lqh

Created

Created container: main
(x15)

kserve-ci-e2e-test

OpenDataHubModelController

scheduler-v06-threshold-migration-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-v06-threshold-migration-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-v06-threshold-migration-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.Secret kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve

kserve-ci-e2e-test

replicaset-controller

scheduler-v06-threshold-migration-test-kserve-5994569d5b

SuccessfulCreate

Created pod: scheduler-v06-threshold-migration-test-kserve-5994569d5b-t8xj9

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.Role kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-role

kserve-ci-e2e-test

deployment-controller

scheduler-v06-threshold-migration-test-kserve

ScalingReplicaSet

Scaled up replica set scheduler-v06-threshold-migration-test-kserve-5994569d5b from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-rb

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.Service kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test0a07e82be2036793543b2d3db2ea0785-scheduler-v0

kserve-ci-e2e-test

replicaset-controller

scheduler-v06-threshold-migration-test-kserve-prefill-5fc599dc49

SuccessfulCreate

Created pod: scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ncl6m

kserve-ci-e2e-test

deployment-controller

scheduler-v06-threshold-migration-test-kserve-prefill

ScalingReplicaSet

Scaled up replica set scheduler-v06-threshold-migration-test-kserve-prefill-5fc599dc49 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

scheduler-v06-threshold-migration-test-kserve-router-scheduler-dbc65f645

SuccessfulCreate

Created pod: scheduler-v06-threshold-migration-test-kserve-router-schedz7ptr

kserve-ci-e2e-test

deployment-controller

scheduler-v06-threshold-migration-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set scheduler-v06-threshold-migration-test-kserve-router-scheduler-dbc65f645 from 0 to 1

kserve-ci-e2e-test

multus

scheduler-v06-threshold-migration-test-kserve-5994569d5b-t8xj9

AddedInterface

Add eth0 [10.133.0.52/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-t8xj9

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-t8xj9

Created

Created container: llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-t8xj9

Pulled

Container image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

multus

scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ncl6m

AddedInterface

Add eth0 [10.133.0.53/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-router-schedz7ptr

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-router-schedz7ptr

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-router-schedz7ptr

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-t8xj9

Started

Started container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-t8xj9

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ncl6m

Started

Started container main

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ncl6m

Created

Created container: main

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ncl6m

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

multus

scheduler-v06-threshold-migration-test-kserve-router-schedz7ptr

AddedInterface

Add eth0 [10.133.0.54/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-t8xj9

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Updated

Updated v1.Secret kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-route
(x10)

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

LLMInferenceServiceReady

LLMInferenceService [scheduler-v06-threshold-migration-test] is Ready
(x11)

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-5f875cbf68-x2lqh

Unhealthy

Startup probe failed: Get "https://10.132.0.64:8000/health": dial tcp 10.132.0.64:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

scheduler-v06-threshold-migration-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test0a07e82be2036793543b2d3db2ea0785-scheduler-v0

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-t8xj9

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-5994569d5b-t8xj9

Killing

Stopping container llm-d-routing-sidecar

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-router-schedz7ptr

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ncl6m

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

scheduler-v06-threshold-migration-test-kserve-router-schedz7ptr

Killing

Stopping container main
(x2)

kserve-ci-e2e-test

LLMInferenceServiceController

stop-feature-test

LLMInferenceServiceReady

LLMInferenceService [stop-feature-test] is Ready

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-5f875cbf68-x2lqh

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-76cf8bfb8f-qgsf9

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-router-scheduler-76cf8bfb8f-qgsf9

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

stop-feature-test-kserve-5f875cbf68-x2lqh

Unhealthy

Readiness probe failed: Get "https://10.132.0.64:8000/health": dial tcp 10.132.0.64:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.Secret kserve-ci-e2e-test/prestop-hook-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-65cf7fd568-kgw6w

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-prestop-hook-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/prestop-hook-test-epp-sa

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-65cf7fd568-kgw6w

Started

Started container storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.Service kserve-ci-e2e-test/prestop-hook-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.Deployment kserve-ci-e2e-test/prestop-hook-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.Service kserve-ci-e2e-test/prestop-hook-test-epp-service

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-65cf7fd568-kgw6w

Created

Created container: storage-initializer

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.Role kserve-ci-e2e-test/prestop-hook-test-epp-role

kserve-ci-e2e-test

multus

prestop-hook-test-kserve-65cf7fd568-kgw6w

AddedInterface

Add eth0 [10.132.0.66/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

prestop-hook-test-kserve-65cf7fd568

SuccessfulCreate

Created pod: prestop-hook-test-kserve-65cf7fd568-kgw6w

kserve-ci-e2e-test

deployment-controller

prestop-hook-test-kserve

ScalingReplicaSet

Scaled up replica set prestop-hook-test-kserve-65cf7fd568 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/prestop-hook-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

Created v1.Deployment kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler

kserve-ci-e2e-test

deployment-controller

prestop-hook-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set prestop-hook-test-kserve-router-scheduler-86bc684b85 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

prestop-hook-test-kserve-router-scheduler-86bc684b85

SuccessfulCreate

Created pod: prestop-hook-test-kserve-router-scheduler-86bc684b85-qqhf2

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-86bc684b85-qqhf2

Created

Created container: storage-initializer

kserve-ci-e2e-test

multus

prestop-hook-test-kserve-router-scheduler-86bc684b85-qqhf2

AddedInterface

Add eth0 [10.132.0.67/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-86bc684b85-qqhf2

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-86bc684b85-qqhf2

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-86bc684b85-qqhf2

Started

Started container main

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-86bc684b85-qqhf2

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-86bc684b85-qqhf2

Created

Created container: main

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-65cf7fd568-kgw6w

Started

Started container main

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-65cf7fd568-kgw6w

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-65cf7fd568-kgw6w

Created

Created container: main
(x15)

kserve-ci-e2e-test

OpenDataHubModelController

lora-single-adapter-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy lora-single-adapter-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/lora-single-adapter-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "lora-single-adapter-test-kserve-route" not found

kserve-ci-e2e-test

replicaset-controller

lora-single-adapter-test-kserve-5849bd8489

SuccessfulCreate

Created pod: lora-single-adapter-test-kserve-5849bd8489-gtjrb

kserve-ci-e2e-test

deployment-controller

lora-single-adapter-test-kserve

ScalingReplicaSet

Scaled up replica set lora-single-adapter-test-kserve-5849bd8489 from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

lora-single-adapter-test

Created

Created v1.Secret kserve-ci-e2e-test/lora-single-adapter-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

lora-single-adapter-test

Created

Created v1.HTTPRoute kserve-ci-e2e-test/lora-single-adapter-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

lora-single-adapter-test

Created

Created v1.Service kserve-ci-e2e-test/lora-single-adapter-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

lora-single-adapter-test

Created

Created v1.Deployment kserve-ci-e2e-test/lora-single-adapter-test-kserve

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-5849bd8489-gtjrb

Created

Created container: storage-initializer

kserve-ci-e2e-test

multus

lora-single-adapter-test-kserve-5849bd8489-gtjrb

AddedInterface

Add eth0 [10.132.0.68/23] from ovn-kubernetes

kserve-ci-e2e-test

LLMInferenceServiceController

lora-single-adapter-test

Created

Created v1.DestinationRule kserve-ci-e2e-test/lora-single-adapter-test-kserve-workload-svc

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-5849bd8489-gtjrb

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-5849bd8489-gtjrb

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-5849bd8489-gtjrb

Started

Started container main

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-5849bd8489-gtjrb

Created

Created container: main

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-5849bd8489-gtjrb

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/prestop-hook-test-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/prestop-hook-test-kserve-shadow-svc
(x11)

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-65cf7fd568-kgw6w

Unhealthy

Startup probe failed: Get "https://10.132.0.66:8000/health": dial tcp 10.132.0.66:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

lora-single-adapter-test

Updated

Updated v1.Secret kserve-ci-e2e-test/lora-single-adapter-test-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-65cf7fd568-dnqbs

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-65cf7fd568-dnqbs

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-65cf7fd568-kgw6w

Killing

Stopping container main

kserve-ci-e2e-test

replicaset-controller

prestop-hook-test-kserve-65cf7fd568

SuccessfulCreate

Created pod: prestop-hook-test-kserve-65cf7fd568-dnqbs

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-65cf7fd568-dnqbs

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

multus

prestop-hook-test-kserve-65cf7fd568-dnqbs

AddedInterface

Add eth0 [10.134.0.64/23] from ovn-kubernetes
(x8)

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-5849bd8489-gtjrb

Unhealthy

Startup probe failed: Get "https://10.132.0.68:8000/health": dial tcp 10.132.0.68:8000: connect: connection refused

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-65cf7fd568-dnqbs

Created

Created container: main

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-65cf7fd568-dnqbs

Started

Started container main

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-65cf7fd568-dnqbs

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

lora-single-adapter-test

LLMInferenceServiceReady

LLMInferenceService [lora-single-adapter-test] is Ready

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-65cf7fd568-kgw6w

Unhealthy

Readiness probe failed: Get "https://10.132.0.66:8000/health": dial tcp 10.132.0.66:8000: connect: connection refused
(x2)

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Updated

Updated v1.Secret kserve-ci-e2e-test/prestop-hook-test-kserve-self-signed-certs
(x33)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs
(x7)

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-65cf7fd568-dnqbs

Unhealthy

Startup probe failed: Get "https://10.134.0.64:8000/health": dial tcp 10.134.0.64:8000: connect: connection refused
(x70)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisvb19f98874e050eec8ca94d49676113f0-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

prestop-hook-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-prestop-hook-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

lora-single-adapter-test-kserve-5849bd8489-gtjrb

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-65cf7fd568-dnqbs

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-86bc684b85-qqhf2

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

prestop-hook-test-kserve-router-scheduler-86bc684b85-qqhf2

FailedPreStopHook

PreStopHook failed
(x14)

kserve-ci-e2e-test

OpenDataHubModelController

lora-multiple-adapters-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy lora-multiple-adapters-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "lora-multiple-adapters-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

lora-multiple-adapters-test

Created

Created v1.Deployment kserve-ci-e2e-test/lora-multiple-adapters-test-kserve

kserve-ci-e2e-test

replicaset-controller

lora-multiple-adapters-test-kserve-66df458ff4

SuccessfulCreate

Created pod: lora-multiple-adapters-test-kserve-66df458ff4-5ntmf

kserve-ci-e2e-test

LLMInferenceServiceController

lora-multiple-adapters-test

Created

Created v1.DestinationRule kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

lora-multiple-adapters-test

Created

Created v1.HTTPRoute kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

lora-multiple-adapters-test

Created

Created v1.Service kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

lora-multiple-adapters-test

Created

Created v1.Secret kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-self-signed-certs

kserve-ci-e2e-test

deployment-controller

lora-multiple-adapters-test-kserve

ScalingReplicaSet

Scaled up replica set lora-multiple-adapters-test-kserve-66df458ff4 from 0 to 1

kserve-ci-e2e-test

multus

lora-multiple-adapters-test-kserve-66df458ff4-5ntmf

AddedInterface

Add eth0 [10.132.0.69/23] from ovn-kubernetes
(x2)

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-66df458ff4-5ntmf

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine
(x2)

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-66df458ff4-5ntmf

Started

Started container storage-initializer
(x2)

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-66df458ff4-5ntmf

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-66df458ff4-5ntmf

Started

Started container main

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-66df458ff4-5ntmf

Created

Created container: main

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-66df458ff4-5ntmf

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

lora-multiple-adapters-test

Updated

Updated v1.Secret kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.Deployment kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler

kserve-ci-e2e-test

replicaset-controller

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd

SuccessfulCreate

Created pod: rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-dfnlm

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.Service kserve-ci-e2e-test/rolling-upgrade-test-epp-service

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.Service kserve-ci-e2e-test/rolling-upgrade-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/rolling-upgrade-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.Role kserve-ci-e2e-test/rolling-upgrade-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-rolling-upgrade-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.Deployment kserve-ci-e2e-test/rolling-upgrade-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.Secret kserve-ci-e2e-test/rolling-upgrade-test-kserve-self-signed-certs
(x15)

kserve-ci-e2e-test

OpenDataHubModelController

rolling-upgrade-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy rolling-upgrade-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/rolling-upgrade-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "rolling-upgrade-test-kserve-route" not found

kserve-ci-e2e-test

replicaset-controller

rolling-upgrade-test-kserve-54bc76f55b

SuccessfulCreate

Created pod: rolling-upgrade-test-kserve-54bc76f55b-xjzqz

kserve-ci-e2e-test

deployment-controller

rolling-upgrade-test-kserve

ScalingReplicaSet

Scaled up replica set rolling-upgrade-test-kserve-54bc76f55b from 0 to 1

kserve-ci-e2e-test

deployment-controller

rolling-upgrade-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd from 0 to 1

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/rolling-upgrade-test-epp-sa

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-54bc76f55b-xjzqz

Started

Started container main

kserve-ci-e2e-test

multus

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-dfnlm

AddedInterface

Add eth0 [10.133.0.56/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-54bc76f55b-xjzqz

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-54bc76f55b-xjzqz

Created

Created container: main

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-dfnlm

Created

Created container: main

kserve-ci-e2e-test

multus

rolling-upgrade-test-kserve-54bc76f55b-xjzqz

AddedInterface

Add eth0 [10.133.0.55/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-dfnlm

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-dfnlm

Started

Started container main
(x7)

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-66df458ff4-5ntmf

Unhealthy

Startup probe failed: Get "https://10.132.0.69:8000/health": dial tcp 10.132.0.69:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/rolling-upgrade-test-kserve-route

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Updated

Updated v1.Secret kserve-ci-e2e-test/rolling-upgrade-test-kserve-self-signed-certs
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/rolling-upgrade-test-kserve-shadow-svc

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

LLMInferenceServiceReady

LLMInferenceService [rolling-upgrade-test] is Ready

kserve-ci-e2e-test

LLMInferenceServiceController

lora-multiple-adapters-test

LLMInferenceServiceReady

LLMInferenceService [lora-multiple-adapters-test] is Ready

kserve-ci-e2e-test

replicaset-controller

rolling-upgrade-test-kserve-859df57bcb

SuccessfulCreate

Created pod: rolling-upgrade-test-kserve-859df57bcb-7z6cd

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Updated

Updated v1.Deployment kserve-ci-e2e-test/rolling-upgrade-test-kserve

kserve-ci-e2e-test

deployment-controller

rolling-upgrade-test-kserve

ScalingReplicaSet

Scaled up replica set rolling-upgrade-test-kserve-859df57bcb from 0 to 1

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-859df57bcb-7z6cd

Started

Started container main

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-859df57bcb-7z6cd

Created

Created container: main

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-859df57bcb-7z6cd

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine

kserve-ci-e2e-test

multus

rolling-upgrade-test-kserve-859df57bcb-7z6cd

AddedInterface

Add eth0 [10.133.0.57/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

rolling-upgrade-test-kserve

ScalingReplicaSet

Scaled down replica set rolling-upgrade-test-kserve-54bc76f55b from 1 to 0

kserve-ci-e2e-test

replicaset-controller

rolling-upgrade-test-kserve-54bc76f55b

SuccessfulDelete

Deleted pod: rolling-upgrade-test-kserve-54bc76f55b-xjzqz

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-54bc76f55b-xjzqz

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-dfnlm

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

LLMInferenceServiceController

rolling-upgrade-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-rolling-upgrade-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-dfnlm

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

rolling-upgrade-test-kserve-859df57bcb-7z6cd

Killing

Stopping container main
(x161)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv8bf079eb6eda4debfb4ef5bb7817824c-kserve-self-signed-certs

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-66df458ff4-5ntmf

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Error

Reconciliation failed: failed to reconcile workload: failed to reconcile self-signed certificates secret: failed to reconcile self-signed TLS certificate: failed to get defaults for v1.Secret kserve-ci-e2e-test/llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs: context canceled

kserve-ci-e2e-test

kubelet

lora-multiple-adapters-test-kserve-66df458ff4-5ntmf

Unhealthy

Readiness probe failed: Get "https://10.132.0.69:8000/health": dial tcp 10.132.0.69:8000: connect: connection refused
(x15)

kserve-ci-e2e-test

OpenDataHubModelController

tls-verification-test

ReconcileError

Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy tls-verification-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/tls-verification-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "tls-verification-test-kserve-route" not found

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.Service kserve-ci-e2e-test/tls-verification-test-kserve-workload-svc

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.Role kserve-ci-e2e-test/tls-verification-test-epp-role

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.ClusterRoleBinding /kserve-ci-e2e-test-tls-verification-test-epp-auth-rb

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.ServiceAccount kserve-ci-e2e-test/tls-verification-test-epp-sa

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.RoleBinding kserve-ci-e2e-test/tls-verification-test-epp-rb

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.Deployment kserve-ci-e2e-test/tls-verification-test-kserve

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.Secret kserve-ci-e2e-test/tls-verification-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.Deployment kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler

kserve-ci-e2e-test

deployment-controller

tls-verification-test-kserve

ScalingReplicaSet

Scaled up replica set tls-verification-test-kserve-759794d6b4 from 0 to 1

kserve-ci-e2e-test

deployment-controller

tls-verification-test-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set tls-verification-test-kserve-router-scheduler-5db688dfd5 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

tls-verification-test-kserve-router-scheduler-5db688dfd5

SuccessfulCreate

Created pod: tls-verification-test-kserve-router-scheduler-5db688dfd5-ngfb8

kserve-ci-e2e-test

multus

tls-verification-test-kserve-759794d6b4-9m772

AddedInterface

Add eth0 [10.132.0.70/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-759794d6b4-9m772

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

Created v1.Service kserve-ci-e2e-test/tls-verification-test-epp-service

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-759794d6b4-9m772

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-759794d6b4-9m772

Started

Started container storage-initializer

kserve-ci-e2e-test

replicaset-controller

tls-verification-test-kserve-759794d6b4

SuccessfulCreate

Created pod: tls-verification-test-kserve-759794d6b4-9m772

kserve-ci-e2e-test

multus

tls-verification-test-kserve-router-scheduler-5db688dfd5-ngfb8

AddedInterface

Add eth0 [10.132.0.71/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-5db688dfd5-ngfb8

Pulled

Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-5db688dfd5-ngfb8

Started

Started container storage-initializer

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-5db688dfd5-ngfb8

Pulled

Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-5db688dfd5-ngfb8

Created

Created container: storage-initializer

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-5db688dfd5-ngfb8

Started

Started container main

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-5db688dfd5-ngfb8

Created

Created container: main

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-759794d6b4-9m772

Pulled

Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-759794d6b4-9m772

Started

Started container main

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-759794d6b4-9m772

Created

Created container: main

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Updated

Updated v1.Secret kserve-ci-e2e-test/tls-verification-test-kserve-self-signed-certs

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Updated

Updated v1.HTTPRoute kserve-ci-e2e-test/tls-verification-test-kserve-route
(x6)

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Created

(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/tls-verification-test-kserve-shadow-svc
(x11)

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-759794d6b4-9m772

Unhealthy

Startup probe failed: Get "https://10.132.0.70:8000/health": dial tcp 10.132.0.70:8000: connect: connection refused

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

LLMInferenceServiceReady

LLMInferenceService [tls-verification-test] is Ready

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-759794d6b4-9m772

Killing

Stopping container main

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-5db688dfd5-ngfb8

Killing

Stopping container main

kserve-ci-e2e-test

LLMInferenceServiceController

tls-verification-test

Deleted

Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-tls-verification-test-epp-auth-rb

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-router-scheduler-5db688dfd5-ngfb8

FailedPreStopHook

PreStopHook failed

kserve-ci-e2e-test

kubelet

tls-verification-test-kserve-759794d6b4-9m772

Unhealthy

Readiness probe failed: Get "https://10.132.0.70:8000/health": dial tcp 10.132.0.70:8000: connect: connection refused
(x7)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-59b9d263

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv8bf079eb6eda4debfb4ef5bb7817824c-kserve-self-signed-certs
(x7)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-2577e794

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs
(x7)

kserve-ci-e2e-test

LLMInferenceServiceController

llmisvc-model-pvc-router-manage-e8706282

Updated

Updated v1.Secret kserve-ci-e2e-test/llmisvb19f98874e050eec8ca94d49676113f0-kserve-self-signed-certs