| Time | Namespace | Component | RelatedObject | Reason | Message |
|---|---|---|---|---|---|
kserve-ci-e2e-test |
error-404-isvc-94f66-predictor-76446b9648-x8kt2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-94f66-predictor-76446b9648-x8kt2 to ip-10-0-133-168.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-411bf-predictor-75d76dc5cd-272s7 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-411bf-predictor-75d76dc5cd-272s7 to ip-10-0-133-168.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-022f7-predictor-59674bc84-vx8dn |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-022f7-predictor-59674bc84-vx8dn to ip-10-0-133-168.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-1-predictor-55cb655557-fr524 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-55cb655557-fr524 to ip-10-0-133-168.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-baff1-predictor-58687f74b9-vqxzz |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz to ip-10-0-142-13.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-94f66-predictor-56bb84dc9f-x28j5 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5 to ip-10-0-142-13.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-78d4f-predictor-79467495b8-z9tfb |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-78d4f-predictor-79467495b8-z9tfb to ip-10-0-133-168.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-d60e3-predictor-84f4999646-pbx7h |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-d60e3-predictor-84f4999646-pbx7h to ip-10-0-133-168.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-cb302-predictor-6f58d6db6-sp4rg |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-cb302-predictor-6f58d6db6-sp4rg to ip-10-0-133-168.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-78d4f-predictor-599d66c69f-fkpzj |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj to ip-10-0-142-13.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-5da44-predictor-77d9c58457-8lswh |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh to ip-10-0-142-13.ec2.internal | ||
kserve-ci-e2e-test |
isvc-xgboost-graph-predictor-669d8d6456-svbpc |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-xgboost-graph-predictor-669d8d6456-svbpc to ip-10-0-133-168.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw to ip-10-0-142-13.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-d60e3-5bd89c6854-mzd9g |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-d60e3-5bd89c6854-mzd9g to ip-10-0-133-168.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-baff1-5ddf668d85-fbqdl |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl to ip-10-0-142-13.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-78d4f-7578756596-fctpv |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-78d4f-7578756596-fctpv to ip-10-0-133-168.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq to ip-10-0-142-13.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-d60e3-predictor-96cb78977-5txz5 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5 to ip-10-0-142-13.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-cb302-predictor-9dcf7c765-pmb6t |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t to ip-10-0-142-13.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-022f7-predictor-744f9d8855-rcmvf |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf to ip-10-0-142-13.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-cb302-6d88566858-jszmf |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-cb302-6d88566858-jszmf to ip-10-0-133-168.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-94f66-f58d9dd86-bfvnv |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-94f66-f58d9dd86-bfvnv to ip-10-0-133-168.ec2.internal | ||
kserve-ci-e2e-test |
model-chainer-554844c84b-9c2tm |
Scheduled |
Successfully assigned kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm to ip-10-0-142-13.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-c6170-predictor-59b777545b-kglp9 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-c6170-predictor-59b777545b-kglp9 to ip-10-0-133-168.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2 to ip-10-0-128-123.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-baff1-predictor-6c8f9d8cc4-pwd6j |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-baff1-predictor-6c8f9d8cc4-pwd6j to ip-10-0-133-168.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-f26f3-predictor-7d74978d64-x6tkp |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-f26f3-predictor-7d74978d64-x6tkp to ip-10-0-133-168.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-c6170-7d9d99d4d4-v4zn6 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-c6170-7d9d99d4d4-v4zn6 to ip-10-0-133-168.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-c6170-predictor-76675d7967-6zrmk |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk to ip-10-0-142-13.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-5da44-predictor-98d5465cd-hvgvx |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-5da44-predictor-98d5465cd-hvgvx to ip-10-0-133-168.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-411bf-d968dbbf-78445 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-411bf-d968dbbf-78445 to ip-10-0-133-168.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-5da44-7795d6f48-6llxp |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-5da44-7795d6f48-6llxp to ip-10-0-133-168.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-f26f3-6c59bfdcd4-vgtln |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-f26f3-6c59bfdcd4-vgtln to ip-10-0-133-168.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-022f7-65bb66f8cc-9hzd2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-022f7-65bb66f8cc-9hzd2 to ip-10-0-133-168.ec2.internal | ||
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-baff1-predictor-6c8f9d8cc4 |
SuccessfulCreate |
Created pod: error-404-isvc-baff1-predictor-6c8f9d8cc4-pwd6j | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-55cb655557-fr524 |
Pulling |
Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:ca9e7e1d07975dffacc4b293d655f2a02df4615629916a1cdc022036c5a27a57" | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-2-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-2-predictor-5d696f88d7 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-baff1-predictor-58687f74b9 |
SuccessfulCreate |
Created pod: success-200-isvc-baff1-predictor-58687f74b9-vqxzz | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-1-predictor-55cb655557 |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-1-predictor-55cb655557-fr524 | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-1-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-1-predictor-55cb655557 from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-baff1-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-baff1-predictor-6c8f9d8cc4 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-1-predictor-55cb655557-fr524 |
AddedInterface |
Add eth0 [10.134.0.23/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
success-200-isvc-baff1-predictor-58687f74b9-vqxzz |
AddedInterface |
Add eth0 [10.133.0.28/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-baff1-predictor-58687f74b9-vqxzz |
Pulling |
Pulling image "quay.io/opendatahub/success-200-isvc:odh-pr-1560" | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-baff1-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-baff1-predictor-58687f74b9 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
error-404-isvc-baff1-predictor-6c8f9d8cc4-pwd6j |
AddedInterface |
Add eth0 [10.134.0.24/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
deployment-controller |
isvc-xgboost-graph-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-xgboost-graph-predictor-669d8d6456 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-xgboost-graph-predictor-669d8d6456 |
SuccessfulCreate |
Created pod: isvc-xgboost-graph-predictor-669d8d6456-svbpc | |
kserve-ci-e2e-test |
multus |
isvc-xgboost-graph-predictor-669d8d6456-svbpc |
AddedInterface |
Add eth0 [10.134.0.25/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2 |
AddedInterface |
Add eth0 [10.132.0.23/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-svbpc |
Pulling |
Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:ca9e7e1d07975dffacc4b293d655f2a02df4615629916a1cdc022036c5a27a57" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-baff1-predictor-6c8f9d8cc4-pwd6j |
Pulling |
Pulling image "quay.io/opendatahub/error-404-isvc:odh-pr-1560" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2 |
Pulling |
Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:ca9e7e1d07975dffacc4b293d655f2a02df4615629916a1cdc022036c5a27a57" | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-2-predictor-5d696f88d7 |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2 | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2 |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2 |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:ca9e7e1d07975dffacc4b293d655f2a02df4615629916a1cdc022036c5a27a57" in 3.088s (3.088s including waiting). Image size: 301469622 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2 |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2 |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1560" | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-baff1-predictor-58687f74b9-vqxzz |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-baff1-predictor-58687f74b9-vqxzz |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-baff1-predictor-58687f74b9-vqxzz |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-baff1-predictor-58687f74b9-vqxzz |
Pulled |
Successfully pulled image "quay.io/opendatahub/success-200-isvc:odh-pr-1560" in 12.333s (12.333s including waiting). Image size: 1335718515 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-svbpc |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:ca9e7e1d07975dffacc4b293d655f2a02df4615629916a1cdc022036c5a27a57" in 13.744s (13.744s including waiting). Image size: 301469622 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-svbpc |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-baff1-predictor-6c8f9d8cc4-pwd6j |
Pulled |
Successfully pulled image "quay.io/opendatahub/error-404-isvc:odh-pr-1560" in 13.954s (13.954s including waiting). Image size: 1334725738 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-55cb655557-fr524 |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-svbpc |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-55cb655557-fr524 |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:ca9e7e1d07975dffacc4b293d655f2a02df4615629916a1cdc022036c5a27a57" in 13.947s (13.947s including waiting). Image size: 301469622 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-55cb655557-fr524 |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-baff1-predictor-58687f74b9-vqxzz |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-baff1-predictor-6c8f9d8cc4-pwd6j |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-baff1-predictor-6c8f9d8cc4-pwd6j |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-baff1-predictor-6c8f9d8cc4-pwd6j |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-baff1-predictor-58687f74b9-vqxzz |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-baff1-predictor-58687f74b9-vqxzz |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.286s (2.286s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-baff1-predictor-6c8f9d8cc4-pwd6j |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.467s (2.467s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-baff1-predictor-6c8f9d8cc4-pwd6j |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-baff1-predictor-6c8f9d8cc4-pwd6j |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-svbpc |
Pulling |
Pulling image "kserve/xgbserver:latest" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-55cb655557-fr524 |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1560" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2 |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2 |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1560" in 13.009s (13.009s including waiting). Image size: 1560927157 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2 |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.262s (2.262s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-55cb655557-fr524 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-55cb655557-fr524 |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1560" in 7.069s (7.069s including waiting). Image size: 1560927157 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-55cb655557-fr524 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-55cb655557-fr524 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-55cb655557-fr524 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-55cb655557-fr524 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-svbpc |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-svbpc |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-svbpc |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-svbpc |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-svbpc |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-svbpc |
Pulled |
Successfully pulled image "kserve/xgbserver:latest" in 19.109s (19.109s including waiting). Image size: 1306325243 bytes. | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-baff1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-baff1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-baff1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-baff1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-baff1-predictor-58687f74b9-vqxzz |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.28:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-baff1 |
InferenceServiceReady |
InferenceService [success-200-isvc-baff1] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-baff1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-baff1 |
InferenceServiceReady |
InferenceService [error-404-isvc-baff1] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-baff1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-baff1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-baff1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-baff1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-baff1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-baff1 |
ScalingReplicaSet |
Scaled up replica set switch-graph-baff1-5ddf668d85 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-baff1 |
InferenceGraphReady |
InferenceGraph [switch-graph-baff1] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-baff1-5ddf668d85 |
SuccessfulCreate |
Created pod: switch-graph-baff1-5ddf668d85-fbqdl | |
kserve-ci-e2e-test |
kubelet |
switch-graph-baff1-5ddf668d85-fbqdl |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-baff1-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
switch-graph-baff1-5ddf668d85-fbqdl |
Pulling |
Pulling image "quay.io/opendatahub/kserve-router@sha256:954d644696f600f3aaec773b2a281b1c1b7eb577f87d9494d88d7bc9da2d5a40" | |
kserve-ci-e2e-test |
multus |
switch-graph-baff1-5ddf668d85-fbqdl |
AddedInterface |
Add eth0 [10.133.0.29/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-baff1-5ddf668d85-fbqdl |
Created |
Created container: switch-graph-baff1 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-baff1-5ddf668d85-fbqdl |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-router@sha256:954d644696f600f3aaec773b2a281b1c1b7eb577f87d9494d88d7bc9da2d5a40" in 2.361s (2.361s including waiting). Image size: 218002526 bytes. | |
kserve-ci-e2e-test |
kubelet |
switch-graph-baff1-5ddf668d85-fbqdl |
Started |
Started container switch-graph-baff1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-2] is Ready | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-baff1-predictor-6c8f9d8cc4-pwd6j |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
switch-graph-baff1-5ddf668d85-fbqdl |
Killing |
Stopping container switch-graph-baff1 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-c6170-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-c6170-predictor-59b777545b from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-c6170-predictor-59b777545b |
SuccessfulCreate |
Created pod: error-404-isvc-c6170-predictor-59b777545b-kglp9 | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-c6170-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-c6170-predictor-76675d7967 from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-c6170 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-c6170": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-c6170": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-c6170 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-c6170": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-c6170 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-c6170": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-c6170": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-c6170 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-c6170": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-baff1-predictor-58687f74b9-vqxzz |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-baff1-predictor-58687f74b9-vqxzz |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-c6170-predictor-76675d7967 |
SuccessfulCreate |
Created pod: success-200-isvc-c6170-predictor-76675d7967-6zrmk | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-baff1-predictor-6c8f9d8cc4-pwd6j |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c6170-predictor-59b777545b-kglp9 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c6170-predictor-59b777545b-kglp9 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c6170-predictor-76675d7967-6zrmk |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-baff1-predictor-6c8f9d8cc4-pwd6j |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.24:8643/healthz": dial tcp 10.134.0.24:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c6170-predictor-59b777545b-kglp9 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c6170-predictor-76675d7967-6zrmk |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c6170-predictor-76675d7967-6zrmk |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c6170-predictor-76675d7967-6zrmk |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c6170-predictor-76675d7967-6zrmk |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c6170-predictor-76675d7967-6zrmk |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1560" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-c6170-predictor-76675d7967-6zrmk |
AddedInterface |
Add eth0 [10.133.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c6170-predictor-59b777545b-kglp9 |
Created |
Created container: kube-rbac-proxy | |
| (x8) | kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-svbpc |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.25:8080: connect: connection refused |
kserve-ci-e2e-test |
multus |
error-404-isvc-c6170-predictor-59b777545b-kglp9 |
AddedInterface |
Add eth0 [10.134.0.26/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c6170-predictor-59b777545b-kglp9 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1560" already present on machine | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-baff1-predictor-6c8f9d8cc4-pwd6j |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.24:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c6170-predictor-59b777545b-kglp9 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-baff1-predictor-58687f74b9-vqxzz |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.28:8643/healthz": dial tcp 10.133.0.28:8643: connect: connection refused | |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-1] is Ready | |
| (x10) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
InferenceServiceReady |
InferenceService [isvc-xgboost-graph] is Ready | |
| (x10) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c6170-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c6170-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c6170-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c6170-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-baff1-5ddf668d85-fbqdl |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
model-chainer-554844c84b-9c2tm |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:954d644696f600f3aaec773b2a281b1c1b7eb577f87d9494d88d7bc9da2d5a40" already present on machine | |
kserve-ci-e2e-test |
multus |
model-chainer-554844c84b-9c2tm |
AddedInterface |
Add eth0 [10.133.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
model-chainer-554844c84b-9c2tm |
Created |
Created container: model-chainer | |
kserve-ci-e2e-test |
kubelet |
model-chainer-554844c84b-9c2tm |
Started |
Started container model-chainer | |
kserve-ci-e2e-test |
replicaset-controller |
model-chainer-554844c84b |
SuccessfulCreate |
Created pod: model-chainer-554844c84b-9c2tm | |
kserve-ci-e2e-test |
deployment-controller |
model-chainer |
ScalingReplicaSet |
Scaled up replica set model-chainer-554844c84b from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
model-chainer |
InferenceGraphReady |
InferenceGraph [model-chainer] is Ready | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-c6170-predictor-76675d7967-6zrmk |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.30:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
model-chainer-554844c84b-9c2tm |
Killing |
Stopping container model-chainer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-55cb655557-fr524 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-svbpc |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-svbpc |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-f26f3-predictor-7d74978d64 |
SuccessfulCreate |
Created pod: error-404-isvc-f26f3-predictor-7d74978d64-x6tkp | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-f26f3-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-f26f3-predictor-7d74978d64 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-f26f3-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-f26f3-predictor-7cd54b6f9f from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-f26f3-predictor-7cd54b6f9f |
SuccessfulCreate |
Created pod: success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-55cb655557-fr524 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq |
AddedInterface |
Add eth0 [10.133.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-f26f3-predictor-7d74978d64-x6tkp |
AddedInterface |
Add eth0 [10.134.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1560" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f26f3-predictor-7d74978d64-x6tkp |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1560" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f26f3-predictor-7d74978d64-x6tkp |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f26f3-predictor-7d74978d64-x6tkp |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f26f3-predictor-7d74978d64-x6tkp |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f26f3-predictor-7d74978d64-x6tkp |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f26f3-predictor-7d74978d64-x6tkp |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-c6170 |
InferenceServiceReady |
InferenceService [success-200-isvc-c6170] is Ready | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2 |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.23:8643/healthz": dial tcp 10.132.0.23:8643: connect: connection refused | |
| (x9) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.23:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-c6170 |
InferenceServiceReady |
InferenceService [error-404-isvc-c6170] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-c6170 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-55cb655557-fr524 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.23:8643/healthz": dial tcp 10.134.0.23:8643: connect: connection refused | |
| (x10) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-55cb655557-fr524 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.23:8080: connect: connection refused |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-c6170 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-c6170 |
InferenceGraphReady |
InferenceGraph [switch-graph-c6170] is Ready | |
kserve-ci-e2e-test |
kubelet |
switch-graph-c6170-7d9d99d4d4-v4zn6 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-c6170-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-c6170 |
ScalingReplicaSet |
Scaled up replica set switch-graph-c6170-7d9d99d4d4 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-c6170-7d9d99d4d4 |
SuccessfulCreate |
Created pod: switch-graph-c6170-7d9d99d4d4-v4zn6 | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f26f3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f26f3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
kubelet |
switch-graph-c6170-7d9d99d4d4-v4zn6 |
Pulling |
Pulling image "quay.io/opendatahub/kserve-router@sha256:954d644696f600f3aaec773b2a281b1c1b7eb577f87d9494d88d7bc9da2d5a40" | |
kserve-ci-e2e-test |
multus |
switch-graph-c6170-7d9d99d4d4-v4zn6 |
AddedInterface |
Add eth0 [10.134.0.28/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-c6170-7d9d99d4d4-v4zn6 |
Started |
Started container switch-graph-c6170 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-c6170-7d9d99d4d4-v4zn6 |
Created |
Created container: switch-graph-c6170 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-c6170-7d9d99d4d4-v4zn6 |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-router@sha256:954d644696f600f3aaec773b2a281b1c1b7eb577f87d9494d88d7bc9da2d5a40" in 1.898s (1.898s including waiting). Image size: 218002526 bytes. | |
| (x6) | kserve-ci-e2e-test |
kubelet |
model-chainer-554844c84b-9c2tm |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c6170-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c6170-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-c6170 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c6170-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c6170-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-c6170 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f26f3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f26f3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.32:8080: connect: connection refused |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-c6170 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-c6170 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f26f3 |
InferenceServiceReady |
InferenceService [error-404-isvc-f26f3] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f26f3 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-f26f3 |
InferenceServiceReady |
InferenceService [success-200-isvc-f26f3] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-f26f3 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
kubelet |
sequence-graph-f26f3-6c59bfdcd4-vgtln |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "sequence-graph-f26f3-serving-cert" not found | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-f26f3 |
InferenceGraphReady |
InferenceGraph [sequence-graph-f26f3] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-f26f3 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-f26f3-6c59bfdcd4 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-f26f3-6c59bfdcd4 |
SuccessfulCreate |
Created pod: sequence-graph-f26f3-6c59bfdcd4-vgtln | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-f26f3-6c59bfdcd4-vgtln |
Started |
Started container sequence-graph-f26f3 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-f26f3-6c59bfdcd4-vgtln |
Created |
Created container: sequence-graph-f26f3 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-f26f3-6c59bfdcd4-vgtln |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:954d644696f600f3aaec773b2a281b1c1b7eb577f87d9494d88d7bc9da2d5a40" already present on machine | |
kserve-ci-e2e-test |
multus |
sequence-graph-f26f3-6c59bfdcd4-vgtln |
AddedInterface |
Add eth0 [10.134.0.29/23] from ovn-kubernetes | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f26f3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f26f3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f26f3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f26f3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-f26f3 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-f26f3 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c6170-predictor-59b777545b-kglp9 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-d60e3-predictor-84f4999646 |
SuccessfulCreate |
Created pod: error-404-isvc-d60e3-predictor-84f4999646-pbx7h | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d60e3-predictor-84f4999646-pbx7h |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1560" already present on machine | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d60e3 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-d60e3": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-d60e3": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d60e3 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-d60e3": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c6170-predictor-59b777545b-kglp9 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
switch-graph-c6170-7d9d99d4d4-v4zn6 |
Killing |
Stopping container switch-graph-c6170 | |
kserve-ci-e2e-test |
multus |
success-200-isvc-d60e3-predictor-96cb78977-5txz5 |
AddedInterface |
Add eth0 [10.133.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d60e3-predictor-96cb78977-5txz5 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1560" already present on machine | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d60e3 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-d60e3": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d60e3 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-d60e3": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-d60e3": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d60e3-predictor-96cb78977-5txz5 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c6170-predictor-76675d7967-6zrmk |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c6170-predictor-76675d7967-6zrmk |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-d60e3-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-d60e3-predictor-96cb78977 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-d60e3-predictor-96cb78977 |
SuccessfulCreate |
Created pod: success-200-isvc-d60e3-predictor-96cb78977-5txz5 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d60e3-predictor-96cb78977-5txz5 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-d60e3-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-d60e3-predictor-84f4999646 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
error-404-isvc-d60e3-predictor-84f4999646-pbx7h |
AddedInterface |
Add eth0 [10.134.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d60e3-predictor-96cb78977-5txz5 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d60e3-predictor-84f4999646-pbx7h |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d60e3-predictor-84f4999646-pbx7h |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d60e3-predictor-96cb78977-5txz5 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d60e3-predictor-96cb78977-5txz5 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d60e3-predictor-84f4999646-pbx7h |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d60e3-predictor-84f4999646-pbx7h |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d60e3-predictor-84f4999646-pbx7h |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c6170-predictor-59b777545b-kglp9 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.26:8643/healthz": dial tcp 10.134.0.26:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-c6170-predictor-59b777545b-kglp9 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.26:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-c6170-7d9d99d4d4-v4zn6 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d60e3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d60e3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d60e3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d60e3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-d60e3-predictor-96cb78977-5txz5 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.33:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-d60e3-predictor-84f4999646-pbx7h |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.30:8080: connect: connection refused |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-5da44-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-5da44-predictor-98d5465cd from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-5da44-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-5da44-predictor-77d9c58457 from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-5da44 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-5da44": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-f26f3-6c59bfdcd4-vgtln |
Killing |
Stopping container sequence-graph-f26f3 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-5da44 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-5da44": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-5da44 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-5da44": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-5da44": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f26f3-predictor-7d74978d64-x6tkp |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f26f3-predictor-7d74978d64-x6tkp |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-5da44 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-5da44": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-5da44": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-5da44-predictor-77d9c58457 |
SuccessfulCreate |
Created pod: success-200-isvc-5da44-predictor-77d9c58457-8lswh | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-5da44-predictor-98d5465cd |
SuccessfulCreate |
Created pod: error-404-isvc-5da44-predictor-98d5465cd-hvgvx | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5da44-predictor-77d9c58457-8lswh |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1560" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5da44-predictor-77d9c58457-8lswh |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5da44-predictor-77d9c58457-8lswh |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5da44-predictor-77d9c58457-8lswh |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5da44-predictor-77d9c58457-8lswh |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5da44-predictor-77d9c58457-8lswh |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5da44-predictor-98d5465cd-hvgvx |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5da44-predictor-98d5465cd-hvgvx |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5da44-predictor-98d5465cd-hvgvx |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5da44-predictor-98d5465cd-hvgvx |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5da44-predictor-98d5465cd-hvgvx |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5da44-predictor-98d5465cd-hvgvx |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1560" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-5da44-predictor-98d5465cd-hvgvx |
AddedInterface |
Add eth0 [10.134.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
success-200-isvc-5da44-predictor-77d9c58457-8lswh |
AddedInterface |
Add eth0 [10.133.0.34/23] from ovn-kubernetes | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d60e3 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d60e3 |
InferenceServiceReady |
InferenceService [success-200-isvc-d60e3] is Ready | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f26f3-predictor-7d74978d64-x6tkp |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.27:8643/healthz": dial tcp 10.134.0.27:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-f26f3-predictor-7d74978d64-x6tkp |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.27:8080: connect: connection refused |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d60e3 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d60e3 |
InferenceServiceReady |
InferenceService [error-404-isvc-d60e3] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-d60e3 |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-d60e3-5bd89c6854 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-d60e3-5bd89c6854 |
SuccessfulCreate |
Created pod: ensemble-graph-d60e3-5bd89c6854-mzd9g | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d60e3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d60e3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-d60e3-5bd89c6854-mzd9g |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "ensemble-graph-d60e3-serving-cert" not found | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-d60e3 |
InferenceGraphReady |
InferenceGraph [ensemble-graph-d60e3] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d60e3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d60e3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-d60e3 |
UpdateFailed |
Failed to update status for InferenceGraph "ensemble-graph-d60e3": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "ensemble-graph-d60e3": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-d60e3 |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "ensemble-graph-d60e3": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-d60e3-5bd89c6854-mzd9g |
Started |
Started container ensemble-graph-d60e3 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-d60e3-5bd89c6854-mzd9g |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:954d644696f600f3aaec773b2a281b1c1b7eb577f87d9494d88d7bc9da2d5a40" already present on machine | |
kserve-ci-e2e-test |
multus |
ensemble-graph-d60e3-5bd89c6854-mzd9g |
AddedInterface |
Add eth0 [10.134.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-d60e3-5bd89c6854-mzd9g |
Created |
Created container: ensemble-graph-d60e3 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-94f66 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-94f66": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-94f66": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-94f66-predictor-56bb84dc9f |
SuccessfulCreate |
Created pod: success-200-isvc-94f66-predictor-56bb84dc9f-x28j5 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-94f66-predictor-76446b9648 |
SuccessfulCreate |
Created pod: error-404-isvc-94f66-predictor-76446b9648-x8kt2 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d60e3-predictor-84f4999646-pbx7h |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-94f66-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-94f66-predictor-56bb84dc9f from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-94f66 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-94f66": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-94f66 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-94f66": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-d60e3-5bd89c6854-mzd9g |
Killing |
Stopping container ensemble-graph-d60e3 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-94f66-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-94f66-predictor-76446b9648 from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-94f66 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-94f66": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-94f66": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d60e3-predictor-96cb78977-5txz5 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d60e3-predictor-96cb78977-5txz5 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d60e3-predictor-84f4999646-pbx7h |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-94f66-predictor-56bb84dc9f-x28j5 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-94f66-predictor-76446b9648-x8kt2 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-94f66-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-94f66-predictor-56bb84dc9f-x28j5 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-94f66-predictor-56bb84dc9f-x28j5 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-94f66-predictor-56bb84dc9f-x28j5 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-94f66-predictor-56bb84dc9f-x28j5 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1560" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-94f66-predictor-56bb84dc9f-x28j5 |
AddedInterface |
Add eth0 [10.133.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-94f66-predictor-56bb84dc9f-x28j5 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-94f66-predictor-76446b9648-x8kt2 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1560" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-94f66-predictor-76446b9648-x8kt2 |
AddedInterface |
Add eth0 [10.134.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-94f66-predictor-76446b9648-x8kt2 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-94f66-predictor-76446b9648-x8kt2 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-94f66-predictor-76446b9648-x8kt2 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-94f66-predictor-76446b9648-x8kt2 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-94f66-predictor-76446b9648-x8kt2 |
Created |
Created container: kube-rbac-proxy | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-f26f3-6c59bfdcd4-vgtln |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d60e3-predictor-96cb78977-5txz5 |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.33:8643/healthz": dial tcp 10.133.0.33:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-5da44-predictor-77d9c58457-8lswh |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.34:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-5da44-predictor-98d5465cd-hvgvx |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.31:8080: connect: connection refused |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-5da44-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-5da44-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-5da44-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-5da44-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-5da44 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-5da44 |
InferenceServiceReady |
InferenceService [success-200-isvc-5da44] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-5da44 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-5da44 |
InferenceServiceReady |
InferenceService [error-404-isvc-5da44] is Ready | |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-d60e3-5bd89c6854-mzd9g |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-94f66-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-94f66-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-5da44-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-5da44-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-5da44-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-5da44-7795d6f48 |
SuccessfulCreate |
Created pod: sequence-graph-5da44-7795d6f48-6llxp | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-5da44-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-5da44 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-5da44-7795d6f48 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-5da44 |
InferenceGraphReady |
InferenceGraph [sequence-graph-5da44] is Ready | |
kserve-ci-e2e-test |
multus |
sequence-graph-5da44-7795d6f48-6llxp |
AddedInterface |
Add eth0 [10.134.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-5da44-7795d6f48-6llxp |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:954d644696f600f3aaec773b2a281b1c1b7eb577f87d9494d88d7bc9da2d5a40" already present on machine | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-5da44-7795d6f48-6llxp |
Created |
Created container: sequence-graph-5da44 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-5da44-7795d6f48-6llxp |
Started |
Started container sequence-graph-5da44 | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-94f66-predictor-56bb84dc9f-x28j5 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.35:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-94f66-predictor-76446b9648-x8kt2 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.33:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5da44-predictor-77d9c58457-8lswh |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5da44-predictor-77d9c58457-8lswh |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-411bf-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-411bf-predictor-75d76dc5cd from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-411bf |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-411bf": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-411bf": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-411bf |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-411bf": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5da44-predictor-98d5465cd-hvgvx |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5da44-predictor-98d5465cd-hvgvx |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-411bf-predictor-75d76dc5cd |
SuccessfulCreate |
Created pod: error-404-isvc-411bf-predictor-75d76dc5cd-272s7 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-5da44-7795d6f48-6llxp |
Killing |
Stopping container sequence-graph-5da44 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-411bf-predictor-85c4db6bcc |
SuccessfulCreate |
Created pod: success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-411bf |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-411bf": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-411bf |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-411bf": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-411bf": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-411bf-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-411bf-predictor-85c4db6bcc from 0 to 1 | |
kserve-ci-e2e-test |
multus |
success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw |
AddedInterface |
Add eth0 [10.133.0.36/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1560" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-411bf-predictor-75d76dc5cd-272s7 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-411bf-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-411bf-predictor-75d76dc5cd-272s7 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-411bf-predictor-75d76dc5cd-272s7 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-411bf-predictor-75d76dc5cd-272s7 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-411bf-predictor-75d76dc5cd-272s7 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-411bf-predictor-75d76dc5cd-272s7 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1560" already present on machine | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-94f66 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-411bf-predictor-75d76dc5cd-272s7 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-411bf-predictor-75d76dc5cd-272s7 |
AddedInterface |
Add eth0 [10.134.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-94f66 |
InferenceServiceReady |
InferenceService [success-200-isvc-94f66] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-94f66 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-94f66 |
InferenceServiceReady |
InferenceService [error-404-isvc-94f66] is Ready | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5da44-predictor-77d9c58457-8lswh |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.34:8643/healthz": dial tcp 10.133.0.34:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5da44-predictor-98d5465cd-hvgvx |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.31:8643/healthz": dial tcp 10.134.0.31:8643: connect: connection refused | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-94f66 |
InferenceGraphReady |
InferenceGraph [ensemble-graph-94f66] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-94f66-f58d9dd86 |
SuccessfulCreate |
Created pod: ensemble-graph-94f66-f58d9dd86-bfvnv | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-94f66 |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-94f66-f58d9dd86 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-94f66-f58d9dd86-bfvnv |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "ensemble-graph-94f66-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-94f66-f58d9dd86-bfvnv |
Created |
Created container: ensemble-graph-94f66 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-94f66-f58d9dd86-bfvnv |
Started |
Started container ensemble-graph-94f66 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-94f66-f58d9dd86-bfvnv |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:954d644696f600f3aaec773b2a281b1c1b7eb577f87d9494d88d7bc9da2d5a40" already present on machine | |
kserve-ci-e2e-test |
multus |
ensemble-graph-94f66-f58d9dd86-bfvnv |
AddedInterface |
Add eth0 [10.134.0.36/23] from ovn-kubernetes | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-5da44-7795d6f48-6llxp |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-94f66-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-94f66-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-94f66-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-94f66-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-411bf-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-411bf-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.36:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-411bf-predictor-75d76dc5cd-272s7 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.35:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-411bf-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-411bf-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-411bf |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-411bf |
InferenceServiceReady |
InferenceService [error-404-isvc-411bf] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-411bf |
InferenceServiceReady |
InferenceService [success-200-isvc-411bf] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-411bf |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-94f66 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-94f66 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-411bf-d968dbbf |
SuccessfulCreate |
Created pod: sequence-graph-411bf-d968dbbf-78445 | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-411bf |
ScalingReplicaSet |
Scaled up replica set sequence-graph-411bf-d968dbbf from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-411bf |
InferenceGraphReady |
InferenceGraph [sequence-graph-411bf] is Ready | |
kserve-ci-e2e-test |
multus |
sequence-graph-411bf-d968dbbf-78445 |
AddedInterface |
Add eth0 [10.134.0.37/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-411bf-d968dbbf-78445 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : failed to sync secret cache: timed out waiting for the condition | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-411bf-d968dbbf-78445 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:954d644696f600f3aaec773b2a281b1c1b7eb577f87d9494d88d7bc9da2d5a40" already present on machine | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-411bf-d968dbbf-78445 |
Created |
Created container: sequence-graph-411bf | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-411bf-d968dbbf-78445 |
Started |
Started container sequence-graph-411bf | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-411bf-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-411bf-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-411bf-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-411bf-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-411bf |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-411bf |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-022f7 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-022f7": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-022f7 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-022f7": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-022f7-predictor-744f9d8855-rcmvf |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-022f7-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-022f7 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-022f7": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-022f7": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-022f7-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-022f7-predictor-59674bc84 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-94f66-predictor-56bb84dc9f-x28j5 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-94f66-predictor-56bb84dc9f-x28j5 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-022f7-predictor-59674bc84 |
SuccessfulCreate |
Created pod: error-404-isvc-022f7-predictor-59674bc84-vx8dn | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-94f66-predictor-76446b9648-x8kt2 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-94f66-predictor-76446b9648-x8kt2 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-022f7-predictor-744f9d8855 |
SuccessfulCreate |
Created pod: success-200-isvc-022f7-predictor-744f9d8855-rcmvf | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-022f7-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-022f7-predictor-744f9d8855 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-94f66-f58d9dd86-bfvnv |
Killing |
Stopping container ensemble-graph-94f66 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-022f7 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-022f7": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-022f7": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-022f7-predictor-744f9d8855-rcmvf |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-022f7-predictor-59674bc84-vx8dn |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-022f7-predictor-59674bc84-vx8dn |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-022f7-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-022f7-predictor-744f9d8855-rcmvf |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-022f7-predictor-59674bc84-vx8dn |
AddedInterface |
Add eth0 [10.134.0.38/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-022f7-predictor-59674bc84-vx8dn |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1560" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-022f7-predictor-744f9d8855-rcmvf |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-022f7-predictor-59674bc84-vx8dn |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-022f7-predictor-59674bc84-vx8dn |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-022f7-predictor-59674bc84-vx8dn |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-022f7-predictor-59674bc84-vx8dn |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-022f7-predictor-744f9d8855-rcmvf |
AddedInterface |
Add eth0 [10.133.0.38/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-022f7-predictor-744f9d8855-rcmvf |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1560" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-022f7-predictor-744f9d8855-rcmvf |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-022f7-predictor-744f9d8855-rcmvf |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-94f66-predictor-56bb84dc9f-x28j5 |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.35:8643/healthz": context deadline exceeded | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-94f66-predictor-56bb84dc9f-x28j5 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.35:8080: i/o timeout | |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-94f66-f58d9dd86-bfvnv |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-022f7-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-022f7-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-022f7-predictor-59674bc84-vx8dn |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.38:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-022f7-predictor-744f9d8855-rcmvf |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.38:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-78d4f |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-78d4f": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-78d4f": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-78d4f-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-78d4f-predictor-599d66c69f from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-78d4f-predictor-599d66c69f |
SuccessfulCreate |
Created pod: success-200-isvc-78d4f-predictor-599d66c69f-fkpzj | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-78d4f-predictor-79467495b8 |
SuccessfulCreate |
Created pod: error-404-isvc-78d4f-predictor-79467495b8-z9tfb | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-78d4f |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-78d4f": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-78d4f |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-78d4f": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-411bf-predictor-75d76dc5cd-272s7 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-411bf-predictor-75d76dc5cd-272s7 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-78d4f |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-78d4f": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-78d4f": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-78d4f-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-78d4f-predictor-79467495b8 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-411bf-d968dbbf-78445 |
Killing |
Stopping container sequence-graph-411bf | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-78d4f-predictor-79467495b8-z9tfb |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-78d4f-predictor-79467495b8-z9tfb |
AddedInterface |
Add eth0 [10.134.0.39/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-78d4f-predictor-599d66c69f-fkpzj |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-78d4f-predictor-79467495b8-z9tfb |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-78d4f-predictor-79467495b8-z9tfb |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-78d4f-predictor-79467495b8-z9tfb |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-78d4f-predictor-599d66c69f-fkpzj |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-78d4f-predictor-599d66c69f-fkpzj |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-78d4f-predictor-599d66c69f-fkpzj |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-78d4f-predictor-599d66c69f-fkpzj |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1560" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-78d4f-predictor-79467495b8-z9tfb |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1560" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-78d4f-predictor-599d66c69f-fkpzj |
AddedInterface |
Add eth0 [10.133.0.39/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-78d4f-predictor-599d66c69f-fkpzj |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-78d4f-predictor-79467495b8-z9tfb |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-022f7 |
InferenceServiceReady |
InferenceService [error-404-isvc-022f7] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-022f7 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-022f7 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-022f7 |
InferenceServiceReady |
InferenceService [success-200-isvc-022f7] is Ready | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-022f7 |
InferenceGraphReady |
InferenceGraph [splitter-graph-022f7] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-022f7-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-022f7-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-022f7 |
ScalingReplicaSet |
Scaled up replica set splitter-graph-022f7-65bb66f8cc from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-022f7-65bb66f8cc |
SuccessfulCreate |
Created pod: splitter-graph-022f7-65bb66f8cc-9hzd2 | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-022f7-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-022f7-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
splitter-graph-022f7-65bb66f8cc-9hzd2 |
Created |
Created container: splitter-graph-022f7 | |
kserve-ci-e2e-test |
multus |
splitter-graph-022f7-65bb66f8cc-9hzd2 |
AddedInterface |
Add eth0 [10.134.0.40/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-022f7-65bb66f8cc-9hzd2 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:954d644696f600f3aaec773b2a281b1c1b7eb577f87d9494d88d7bc9da2d5a40" already present on machine | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-022f7-65bb66f8cc-9hzd2 |
Started |
Started container splitter-graph-022f7 | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-022f7-65bb66f8cc-9hzd2 |
Killing |
Stopping container splitter-graph-022f7 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-022f7-predictor-59674bc84-vx8dn |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-022f7-predictor-59674bc84-vx8dn |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-cb302-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-cb302-predictor-6f58d6db6 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
error-404-isvc-cb302-predictor-6f58d6db6-sp4rg |
AddedInterface |
Add eth0 [10.134.0.41/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-cb302-predictor-9dcf7c765-pmb6t |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-cb302-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-cb302-predictor-6f58d6db6-sp4rg |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-cb302 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-cb302": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-cb302-predictor-6f58d6db6 |
SuccessfulCreate |
Created pod: error-404-isvc-cb302-predictor-6f58d6db6-sp4rg | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-cb302-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-cb302-predictor-9dcf7c765 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-cb302-predictor-6f58d6db6-sp4rg |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-cb302 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-cb302": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-cb302": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-cb302 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-cb302": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-cb302-predictor-6f58d6db6-sp4rg |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-cb302-predictor-6f58d6db6-sp4rg |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1560" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-cb302-predictor-6f58d6db6-sp4rg |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-cb302 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-cb302": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-cb302": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-022f7-predictor-744f9d8855-rcmvf |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-cb302-predictor-6f58d6db6-sp4rg |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-022f7-predictor-744f9d8855-rcmvf |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-cb302-predictor-9dcf7c765 |
SuccessfulCreate |
Created pod: success-200-isvc-cb302-predictor-9dcf7c765-pmb6t | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-cb302-predictor-9dcf7c765-pmb6t |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-cb302-predictor-9dcf7c765-pmb6t |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-cb302-predictor-9dcf7c765-pmb6t |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-cb302-predictor-9dcf7c765-pmb6t |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1560" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-cb302-predictor-9dcf7c765-pmb6t |
AddedInterface |
Add eth0 [10.133.0.40/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-cb302-predictor-9dcf7c765-pmb6t |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-cb302-predictor-9dcf7c765-pmb6t |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-022f7-predictor-59674bc84-vx8dn |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.38:8643/healthz": dial tcp 10.134.0.38:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-411bf-d968dbbf-78445 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-78d4f-predictor-599d66c69f-fkpzj |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.39:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-78d4f-predictor-79467495b8-z9tfb |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.39:8080: connect: connection refused |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-cb302-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-cb302-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-78d4f |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-78d4f |
InferenceServiceReady |
InferenceService [success-200-isvc-78d4f] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-78d4f |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-78d4f |
InferenceServiceReady |
InferenceService [error-404-isvc-78d4f] is Ready | |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-022f7-65bb66f8cc-9hzd2 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-cb302-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-cb302-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-78d4f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-78d4f-7578756596 |
SuccessfulCreate |
Created pod: switch-graph-78d4f-7578756596-fctpv | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-78d4f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-78d4f |
InferenceGraphReady |
InferenceGraph [switch-graph-78d4f] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-78d4f |
ScalingReplicaSet |
Scaled up replica set switch-graph-78d4f-7578756596 from 0 to 1 | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-78d4f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-78d4f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
switch-graph-78d4f-7578756596-fctpv |
Created |
Created container: switch-graph-78d4f | |
kserve-ci-e2e-test |
kubelet |
switch-graph-78d4f-7578756596-fctpv |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:954d644696f600f3aaec773b2a281b1c1b7eb577f87d9494d88d7bc9da2d5a40" already present on machine | |
kserve-ci-e2e-test |
kubelet |
switch-graph-78d4f-7578756596-fctpv |
Started |
Started container switch-graph-78d4f | |
kserve-ci-e2e-test |
kubelet |
switch-graph-78d4f-7578756596-fctpv |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-78d4f-serving-cert" not found | |
kserve-ci-e2e-test |
multus |
switch-graph-78d4f-7578756596-fctpv |
AddedInterface |
Add eth0 [10.134.0.42/23] from ovn-kubernetes | |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-cb302-predictor-6f58d6db6-sp4rg |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.41:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-cb302-predictor-9dcf7c765-pmb6t |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.40:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-cb302 |
InferenceServiceReady |
InferenceService [success-200-isvc-cb302] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-cb302 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-cb302 |
InferenceServiceReady |
InferenceService [error-404-isvc-cb302] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-cb302 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-78d4f |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-78d4f |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
multus |
splitter-graph-cb302-6d88566858-jszmf |
AddedInterface |
Add eth0 [10.134.0.43/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-cb302-6d88566858-jszmf |
Created |
Created container: splitter-graph-cb302 | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-cb302-6d88566858-jszmf |
Started |
Started container splitter-graph-cb302 | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-cb302 |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "splitter-graph-cb302": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-cb302-6d88566858-jszmf |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:954d644696f600f3aaec773b2a281b1c1b7eb577f87d9494d88d7bc9da2d5a40" already present on machine | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-cb302-6d88566858 |
SuccessfulCreate |
Created pod: splitter-graph-cb302-6d88566858-jszmf | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-cb302 |
InferenceGraphReady |
InferenceGraph [splitter-graph-cb302] is Ready | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-cb302 |
UpdateFailed |
Failed to update status for InferenceGraph "splitter-graph-cb302": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "splitter-graph-cb302": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-cb302 |
ScalingReplicaSet |
Scaled up replica set splitter-graph-cb302-6d88566858 from 0 to 1 | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-78d4f |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-78d4f |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-cb302-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-cb302-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-cb302-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-cb302-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-cb302 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-cb302 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
splitter-graph-cb302-6d88566858-jszmf |
Killing |
Stopping container splitter-graph-cb302 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-cb302-predictor-9dcf7c765-pmb6t |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-cb302-predictor-9dcf7c765-pmb6t |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-cb302-predictor-6f58d6db6-sp4rg |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-cb302-predictor-6f58d6db6-sp4rg |
Killing |
Stopping container kserve-container | |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-cb302-6d88566858-jszmf |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
switch-graph-78d4f-7578756596-fctpv |
Killing |
Stopping container switch-graph-78d4f | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-78d4f-predictor-79467495b8-z9tfb |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-78d4f-predictor-79467495b8-z9tfb |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-78d4f-predictor-599d66c69f-fkpzj |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-78d4f-predictor-599d66c69f-fkpzj |
Killing |
Stopping container kube-rbac-proxy | |
| (x5) | kserve-ci-e2e-test |
kubelet |
switch-graph-78d4f-7578756596-fctpv |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |