| Time | Namespace | Component | RelatedObject | Reason | Message |
|---|---|---|---|---|---|
kserve-ci-e2e-test |
sequence-graph-94d00-868df69ffb-5rp86 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-94d00-868df69ffb-5rp86 to ip-10-0-128-114.ec2.internal | ||
kserve-ci-e2e-test |
model-chainer-644bd56996-mww9p |
Scheduled |
Successfully assigned kserve-ci-e2e-test/model-chainer-644bd56996-mww9p to ip-10-0-133-250.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-a162f-predictor-56c4c966dc-db74g |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g to ip-10-0-136-239.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-8c81a-predictor-756657c85-2pmpb |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb to ip-10-0-136-239.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-d5455-predictor-788d7ccf85-mskgp |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp to ip-10-0-136-239.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-1-predictor-6ff6c549bd-2xhvh |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ff6c549bd-2xhvh to ip-10-0-128-114.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-4a809-predictor-5dcd77f4d6-v75w9 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-4a809-predictor-5dcd77f4d6-v75w9 to ip-10-0-133-250.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-4a809-predictor-6db7b75455-g9d8f |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f to ip-10-0-136-239.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-493ba-predictor-587766b78d-tbfg9 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-493ba-predictor-587766b78d-tbfg9 to ip-10-0-133-250.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd to ip-10-0-136-239.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-315c2-6dffcdf597-99zwl |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-315c2-6dffcdf597-99zwl to ip-10-0-128-114.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-a162f-964d79f84-rgrt2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-a162f-964d79f84-rgrt2 to ip-10-0-128-114.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-4a809-5b56dc99f5-zhq2h |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-4a809-5b56dc99f5-zhq2h to ip-10-0-133-250.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-493ba-59c7bfbd4c-f2ddc |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-493ba-59c7bfbd4c-f2ddc to ip-10-0-128-114.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-01f07-6d5fcffdb-85b7c |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-01f07-6d5fcffdb-85b7c to ip-10-0-133-250.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-d5455-predictor-6776956984-nv48x |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-d5455-predictor-6776956984-nv48x to ip-10-0-133-250.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-a162f-predictor-5955586d98-wskzk |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-a162f-predictor-5955586d98-wskzk to ip-10-0-133-250.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-a1525-predictor-596bfc45bb-kpntg |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-a1525-predictor-596bfc45bb-kpntg to ip-10-0-133-250.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-94d00-predictor-85bcffb47b-j4wcf |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-94d00-predictor-85bcffb47b-j4wcf to ip-10-0-133-250.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-8c81a-predictor-f8d8945c6-v2fbg |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-8c81a-predictor-f8d8945c6-v2fbg to ip-10-0-133-250.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-a1525-5c67bbd599-28jst |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-a1525-5c67bbd599-28jst to ip-10-0-128-114.ec2.internal | ||
kserve-ci-e2e-test |
isvc-xgboost-graph-predictor-555fc69d56-2nfrt |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nfrt to ip-10-0-128-114.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-47ede-predictor-6fd67659b4-bxjfg |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-47ede-predictor-6fd67659b4-bxjfg to ip-10-0-133-250.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv to ip-10-0-136-239.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-315c2-predictor-69cd9f9dfd-h8vcq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-315c2-predictor-69cd9f9dfd-h8vcq to ip-10-0-133-250.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-d5455-58879b467-qdxfr |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-d5455-58879b467-qdxfr to ip-10-0-128-114.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-01f07-predictor-5758564b9f-55hk9 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9 to ip-10-0-136-239.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-2-predictor-cf88c5444-mzkmn |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-cf88c5444-mzkmn to ip-10-0-128-114.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-47ede-d4cf4b666-4btmn |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-47ede-d4cf4b666-4btmn to ip-10-0-128-114.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-8c81a-784498d6cc-9qrmg |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-8c81a-784498d6cc-9qrmg to ip-10-0-128-114.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-01f07-predictor-678d7fdb78-wxkjk |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-01f07-predictor-678d7fdb78-wxkjk to ip-10-0-133-250.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-a1525-predictor-7f846d5f79-7nw6m |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m to ip-10-0-136-239.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd to ip-10-0-136-239.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-94d00-predictor-558f596f5c-5fxt9 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9 to ip-10-0-136-239.ec2.internal | ||
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-1-predictor-6ff6c549bd |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-1-predictor-6ff6c549bd-2xhvh | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-4a809-predictor-6db7b75455 |
SuccessfulCreate |
Created pod: error-404-isvc-4a809-predictor-6db7b75455-g9d8f | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-2-predictor-cf88c5444 |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-2-predictor-cf88c5444-mzkmn | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-2-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-2-predictor-cf88c5444 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6ff6c549bd-2xhvh |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "isvc-sklearn-graph-1-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-4a809-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-4a809-predictor-5dcd77f4d6 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
error-404-isvc-4a809-predictor-6db7b75455-g9d8f |
AddedInterface |
Add eth0 [10.133.0.26/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-xgboost-graph-predictor-555fc69d56 |
SuccessfulCreate |
Created pod: isvc-xgboost-graph-predictor-555fc69d56-2nfrt | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-4a809-predictor-5dcd77f4d6 |
SuccessfulCreate |
Created pod: success-200-isvc-4a809-predictor-5dcd77f4d6-v75w9 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-4a809-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-4a809-predictor-6db7b75455 from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-1-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-1-predictor-6ff6c549bd from 0 to 1 | |
kserve-ci-e2e-test |
multus |
success-200-isvc-4a809-predictor-5dcd77f4d6-v75w9 |
AddedInterface |
Add eth0 [10.132.0.22/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
deployment-controller |
isvc-xgboost-graph-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-xgboost-graph-predictor-555fc69d56 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a809-predictor-5dcd77f4d6-v75w9 |
Pulling |
Pulling image "quay.io/opendatahub/success-200-isvc:odh-pr-1846" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6ff6c549bd-2xhvh |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-1-predictor-6ff6c549bd-2xhvh |
AddedInterface |
Add eth0 [10.134.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2nfrt |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2nfrt |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-2-predictor-cf88c5444-mzkmn |
AddedInterface |
Add eth0 [10.134.0.29/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
isvc-xgboost-graph-predictor-555fc69d56-2nfrt |
AddedInterface |
Add eth0 [10.134.0.28/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-cf88c5444-mzkmn |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2nfrt |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:5d4d7c63778446964149b8ced30c467eba30d56c2030b75a2303be0a7720b701" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6ff6c549bd-2xhvh |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:5d4d7c63778446964149b8ced30c467eba30d56c2030b75a2303be0a7720b701" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6ff6c549bd-2xhvh |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-cf88c5444-mzkmn |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:5d4d7c63778446964149b8ced30c467eba30d56c2030b75a2303be0a7720b701" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a809-predictor-6db7b75455-g9d8f |
Pulling |
Pulling image "quay.io/opendatahub/error-404-isvc:odh-pr-1846" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-cf88c5444-mzkmn |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-cf88c5444-mzkmn |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1846" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6ff6c549bd-2xhvh |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1846" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2nfrt |
Pulling |
Pulling image "kserve/xgbserver:latest" | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a809-predictor-5dcd77f4d6-v75w9 |
Pulled |
Successfully pulled image "quay.io/opendatahub/success-200-isvc:odh-pr-1846" in 11.106s (11.106s including waiting). Image size: 1334932061 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a809-predictor-5dcd77f4d6-v75w9 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a809-predictor-5dcd77f4d6-v75w9 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a809-predictor-5dcd77f4d6-v75w9 |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a809-predictor-6db7b75455-g9d8f |
Pulled |
Successfully pulled image "quay.io/opendatahub/error-404-isvc:odh-pr-1846" in 11.145s (11.145s including waiting). Image size: 1333945432 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a809-predictor-6db7b75455-g9d8f |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a809-predictor-6db7b75455-g9d8f |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a809-predictor-6db7b75455-g9d8f |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a809-predictor-5dcd77f4d6-v75w9 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a809-predictor-5dcd77f4d6-v75w9 |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.086s (2.086s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a809-predictor-5dcd77f4d6-v75w9 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a809-predictor-6db7b75455-g9d8f |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a809-predictor-6db7b75455-g9d8f |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.108s (2.108s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a809-predictor-6db7b75455-g9d8f |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-cf88c5444-mzkmn |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1846" in 28.086s (28.086s including waiting). Image size: 1557915556 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6ff6c549bd-2xhvh |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1846" in 28.089s (28.089s including waiting). Image size: 1557915556 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2nfrt |
Pulled |
Successfully pulled image "kserve/xgbserver:latest" in 28.691s (28.691s including waiting). Image size: 1307355899 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-cf88c5444-mzkmn |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6ff6c549bd-2xhvh |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6ff6c549bd-2xhvh |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6ff6c549bd-2xhvh |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-cf88c5444-mzkmn |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2nfrt |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2nfrt |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2nfrt |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-cf88c5444-mzkmn |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-cf88c5444-mzkmn |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6ff6c549bd-2xhvh |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2nfrt |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.648s (2.648s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-cf88c5444-mzkmn |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 3.172s (3.172s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-cf88c5444-mzkmn |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2nfrt |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6ff6c549bd-2xhvh |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 3.15s (3.15s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6ff6c549bd-2xhvh |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2nfrt |
Started |
Started container kube-rbac-proxy | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4a809-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4a809-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a809-predictor-6db7b75455-g9d8f |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.26:8080: connect: connection refused |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4a809-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4a809-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-4a809 |
InferenceServiceReady |
InferenceService [error-404-isvc-4a809] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-4a809 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-4a809 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-4a809 |
InferenceServiceReady |
InferenceService [success-200-isvc-4a809] is Ready | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4a809-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4a809-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4a809-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4a809-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
multus |
switch-graph-4a809-5b56dc99f5-zhq2h |
AddedInterface |
Add eth0 [10.132.0.23/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-4a809-5b56dc99f5-zhq2h |
Pulling |
Pulling image "quay.io/opendatahub/kserve-router@sha256:65dcd869d2c6fd26e4cba78f3f0eef9e2fb99b132c1879d2b886845544ac1220" | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-4a809-5b56dc99f5 |
SuccessfulCreate |
Created pod: switch-graph-4a809-5b56dc99f5-zhq2h | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-4a809 |
ScalingReplicaSet |
Scaled up replica set switch-graph-4a809-5b56dc99f5 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-4a809 |
InferenceGraphReady |
InferenceGraph [switch-graph-4a809] is Ready | |
kserve-ci-e2e-test |
kubelet |
switch-graph-4a809-5b56dc99f5-zhq2h |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-router@sha256:65dcd869d2c6fd26e4cba78f3f0eef9e2fb99b132c1879d2b886845544ac1220" in 2.582s (2.582s including waiting). Image size: 218252378 bytes. | |
kserve-ci-e2e-test |
kubelet |
switch-graph-4a809-5b56dc99f5-zhq2h |
Started |
Started container switch-graph-4a809 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-4a809-5b56dc99f5-zhq2h |
Created |
Created container: switch-graph-4a809 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-01f07 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-01f07": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-01f07-predictor-678d7fdb78 |
SuccessfulCreate |
Created pod: success-200-isvc-01f07-predictor-678d7fdb78-wxkjk | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a809-predictor-6db7b75455-g9d8f |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a809-predictor-5dcd77f4d6-v75w9 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
switch-graph-4a809-5b56dc99f5-zhq2h |
Killing |
Stopping container switch-graph-4a809 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a809-predictor-6db7b75455-g9d8f |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-01f07-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-01f07-predictor-5758564b9f from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-01f07 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-01f07": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-01f07 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-01f07": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-01f07": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-01f07-predictor-5758564b9f |
SuccessfulCreate |
Created pod: error-404-isvc-01f07-predictor-5758564b9f-55hk9 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a809-predictor-5dcd77f4d6-v75w9 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-01f07 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-01f07": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-01f07": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-01f07-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-01f07-predictor-678d7fdb78 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a809-predictor-5dcd77f4d6-v75w9 |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.22:8643/healthz": dial tcp 10.132.0.22:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-01f07-predictor-5758564b9f-55hk9 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
success-200-isvc-01f07-predictor-678d7fdb78-wxkjk |
AddedInterface |
Add eth0 [10.132.0.24/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-01f07-predictor-678d7fdb78-wxkjk |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1846" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-01f07-predictor-678d7fdb78-wxkjk |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-01f07-predictor-678d7fdb78-wxkjk |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-01f07-predictor-678d7fdb78-wxkjk |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-01f07-predictor-678d7fdb78-wxkjk |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-01f07-predictor-678d7fdb78-wxkjk |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-01f07-predictor-5758564b9f-55hk9 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-01f07-predictor-5758564b9f-55hk9 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-01f07-predictor-5758564b9f-55hk9 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-01f07-predictor-5758564b9f-55hk9 |
Created |
Created container: kserve-container | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a809-predictor-5dcd77f4d6-v75w9 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.22:8080: connect: connection refused |
kserve-ci-e2e-test |
multus |
error-404-isvc-01f07-predictor-5758564b9f-55hk9 |
AddedInterface |
Add eth0 [10.133.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-01f07-predictor-5758564b9f-55hk9 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1846" already present on machine | |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
InferenceServiceReady |
InferenceService [isvc-xgboost-graph] is Ready | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-2] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-1] is Ready | |
| (x10) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-4a809-5b56dc99f5-zhq2h |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-01f07-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-01f07-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-01f07-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-01f07-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-01f07-predictor-678d7fdb78-wxkjk |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.24:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-01f07 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-01f07 |
InferenceServiceReady |
InferenceService [error-404-isvc-01f07] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-01f07 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-01f07 |
InferenceServiceReady |
InferenceService [success-200-isvc-01f07] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
model-chainer |
ScalingReplicaSet |
Scaled up replica set model-chainer-644bd56996 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
model-chainer-644bd56996-mww9p |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "model-chainer-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
model-chainer-644bd56996 |
SuccessfulCreate |
Created pod: model-chainer-644bd56996-mww9p | |
kserve-ci-e2e-test |
InferenceGraphController |
model-chainer |
InferenceGraphReady |
InferenceGraph [model-chainer] is Ready | |
kserve-ci-e2e-test |
kubelet |
model-chainer-644bd56996-mww9p |
Started |
Started container model-chainer | |
kserve-ci-e2e-test |
multus |
model-chainer-644bd56996-mww9p |
AddedInterface |
Add eth0 [10.132.0.25/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
model-chainer-644bd56996-mww9p |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:65dcd869d2c6fd26e4cba78f3f0eef9e2fb99b132c1879d2b886845544ac1220" already present on machine | |
kserve-ci-e2e-test |
kubelet |
model-chainer-644bd56996-mww9p |
Created |
Created container: model-chainer | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2nfrt |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-cf88c5444-mzkmn |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2nfrt |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-315c2-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-315c2-predictor-f5d6df9c5 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
success-200-isvc-315c2-predictor-69cd9f9dfd-h8vcq |
AddedInterface |
Add eth0 [10.132.0.26/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
model-chainer-644bd56996-mww9p |
Killing |
Stopping container model-chainer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-cf88c5444-mzkmn |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-01f07-6d5fcffdb |
SuccessfulCreate |
Created pod: switch-graph-01f07-6d5fcffdb-85b7c | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-315c2-predictor-69cd9f9dfd |
SuccessfulCreate |
Created pod: success-200-isvc-315c2-predictor-69cd9f9dfd-h8vcq | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-315c2-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-315c2-predictor-69cd9f9dfd from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-315c2-predictor-f5d6df9c5 |
SuccessfulCreate |
Created pod: error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6ff6c549bd-2xhvh |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6ff6c549bd-2xhvh |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-01f07 |
InferenceGraphReady |
InferenceGraph [switch-graph-01f07] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-01f07 |
ScalingReplicaSet |
Scaled up replica set switch-graph-01f07-6d5fcffdb from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-315c2-predictor-69cd9f9dfd-h8vcq |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1846" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
switch-graph-01f07-6d5fcffdb-85b7c |
Started |
Started container switch-graph-01f07 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-01f07-6d5fcffdb-85b7c |
Created |
Created container: switch-graph-01f07 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-01f07-6d5fcffdb-85b7c |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:65dcd869d2c6fd26e4cba78f3f0eef9e2fb99b132c1879d2b886845544ac1220" already present on machine | |
kserve-ci-e2e-test |
multus |
switch-graph-01f07-6d5fcffdb-85b7c |
AddedInterface |
Add eth0 [10.132.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd |
AddedInterface |
Add eth0 [10.133.0.28/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-315c2-predictor-69cd9f9dfd-h8vcq |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-315c2-predictor-69cd9f9dfd-h8vcq |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1846" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-315c2-predictor-69cd9f9dfd-h8vcq |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-315c2-predictor-69cd9f9dfd-h8vcq |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-315c2-predictor-69cd9f9dfd-h8vcq |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2nfrt |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.28:8643/healthz": dial tcp 10.134.0.28:8643: connect: connection refused | |
| (x10) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-cf88c5444-mzkmn |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.29:8080: connect: connection refused |
| (x10) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6ff6c549bd-2xhvh |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.27:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6ff6c549bd-2xhvh |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.27:8643/healthz": dial tcp 10.134.0.27:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-cf88c5444-mzkmn |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.29:8643/healthz": dial tcp 10.134.0.29:8643: connect: connection refused | |
| (x9) | kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2nfrt |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.28:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-01f07-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-01f07-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-01f07-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-01f07-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
model-chainer-644bd56996-mww9p |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-01f07 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-01f07 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-315c2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-315c2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-315c2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-315c2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-315c2 |
InferenceServiceReady |
InferenceService [success-200-isvc-315c2] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-315c2 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-315c2 |
InferenceServiceReady |
InferenceService [error-404-isvc-315c2] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-315c2 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-315c2 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-315c2-6dffcdf597 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-315c2 |
InferenceGraphReady |
InferenceGraph [sequence-graph-315c2] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-315c2-6dffcdf597 |
SuccessfulCreate |
Created pod: sequence-graph-315c2-6dffcdf597-99zwl | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-315c2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
sequence-graph-315c2-6dffcdf597-99zwl |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "sequence-graph-315c2-serving-cert" not found | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-315c2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
sequence-graph-315c2-6dffcdf597-99zwl |
Pulling |
Pulling image "quay.io/opendatahub/kserve-router@sha256:65dcd869d2c6fd26e4cba78f3f0eef9e2fb99b132c1879d2b886845544ac1220" | |
kserve-ci-e2e-test |
multus |
sequence-graph-315c2-6dffcdf597-99zwl |
AddedInterface |
Add eth0 [10.134.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-315c2-6dffcdf597-99zwl |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-router@sha256:65dcd869d2c6fd26e4cba78f3f0eef9e2fb99b132c1879d2b886845544ac1220" in 1.794s (1.794s including waiting). Image size: 218252378 bytes. | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-315c2-6dffcdf597-99zwl |
Created |
Created container: sequence-graph-315c2 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-315c2-6dffcdf597-99zwl |
Started |
Started container sequence-graph-315c2 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-315c2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-315c2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-315c2 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-315c2 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-a1525-predictor-596bfc45bb |
SuccessfulCreate |
Created pod: success-200-isvc-a1525-predictor-596bfc45bb-kpntg | |
kserve-ci-e2e-test |
kubelet |
switch-graph-01f07-6d5fcffdb-85b7c |
Killing |
Stopping container switch-graph-01f07 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-a1525-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-a1525-predictor-7f846d5f79 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-01f07-predictor-678d7fdb78-wxkjk |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-01f07-predictor-678d7fdb78-wxkjk |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-a1525 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-a1525": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-01f07-predictor-5758564b9f-55hk9 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-01f07-predictor-5758564b9f-55hk9 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-a1525 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-a1525": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-a1525": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-a1525 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-a1525": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-a1525 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-a1525": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-a1525": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-a1525-predictor-7f846d5f79 |
SuccessfulCreate |
Created pod: error-404-isvc-a1525-predictor-7f846d5f79-7nw6m | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-a1525-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-a1525-predictor-596bfc45bb from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a1525-predictor-596bfc45bb-kpntg |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-a1525-predictor-7f846d5f79-7nw6m |
AddedInterface |
Add eth0 [10.133.0.29/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a1525-predictor-7f846d5f79-7nw6m |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1846" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a1525-predictor-7f846d5f79-7nw6m |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a1525-predictor-7f846d5f79-7nw6m |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a1525-predictor-7f846d5f79-7nw6m |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a1525-predictor-7f846d5f79-7nw6m |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a1525-predictor-7f846d5f79-7nw6m |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-a1525-predictor-596bfc45bb-kpntg |
AddedInterface |
Add eth0 [10.132.0.28/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a1525-predictor-596bfc45bb-kpntg |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1846" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a1525-predictor-596bfc45bb-kpntg |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a1525-predictor-596bfc45bb-kpntg |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a1525-predictor-596bfc45bb-kpntg |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a1525-predictor-596bfc45bb-kpntg |
Started |
Started container kube-rbac-proxy | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-01f07-predictor-5758564b9f-55hk9 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.27:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-01f07-predictor-5758564b9f-55hk9 |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.27:8643/healthz": dial tcp 10.133.0.27:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-01f07-6d5fcffdb-85b7c |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-a1525-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-a1525-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-a1525-predictor-596bfc45bb-kpntg |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.28:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-a1525-predictor-7f846d5f79-7nw6m |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.29:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-a1525-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-a1525-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-a1525 |
InferenceServiceReady |
InferenceService [success-200-isvc-a1525] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-a1525 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-a1525 |
InferenceServiceReady |
InferenceService [error-404-isvc-a1525] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-a1525 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-94d00-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-94d00-predictor-558f596f5c from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-a1525-5c67bbd599 |
SuccessfulCreate |
Created pod: ensemble-graph-a1525-5c67bbd599-28jst | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-315c2-predictor-69cd9f9dfd-h8vcq |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-315c2-6dffcdf597-99zwl |
Killing |
Stopping container sequence-graph-315c2 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-94d00-predictor-85bcffb47b-j4wcf |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-94d00-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-94d00-predictor-85bcffb47b |
SuccessfulCreate |
Created pod: success-200-isvc-94d00-predictor-85bcffb47b-j4wcf | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-94d00-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-94d00-predictor-85bcffb47b from 0 to 1 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-a1525-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-a1525-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-315c2-predictor-69cd9f9dfd-h8vcq |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-94d00-predictor-558f596f5c |
SuccessfulCreate |
Created pod: error-404-isvc-94d00-predictor-558f596f5c-5fxt9 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-a1525-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-a1525-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-a1525-5c67bbd599-28jst |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "ensemble-graph-a1525-serving-cert" not found | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-a1525 |
InferenceGraphReady |
InferenceGraph [ensemble-graph-a1525] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-a1525 |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-a1525-5c67bbd599 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-94d00-predictor-85bcffb47b-j4wcf |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-94d00-predictor-85bcffb47b-j4wcf |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-94d00-predictor-85bcffb47b-j4wcf |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
ensemble-graph-a1525-5c67bbd599-28jst |
AddedInterface |
Add eth0 [10.134.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-a1525-5c67bbd599-28jst |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:65dcd869d2c6fd26e4cba78f3f0eef9e2fb99b132c1879d2b886845544ac1220" already present on machine | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-a1525-5c67bbd599-28jst |
Created |
Created container: ensemble-graph-a1525 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-a1525-5c67bbd599-28jst |
Started |
Started container ensemble-graph-a1525 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-94d00-predictor-85bcffb47b-j4wcf |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-94d00-predictor-558f596f5c-5fxt9 |
AddedInterface |
Add eth0 [10.133.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-94d00-predictor-558f596f5c-5fxt9 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1846" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-94d00-predictor-558f596f5c-5fxt9 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-94d00-predictor-558f596f5c-5fxt9 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-94d00-predictor-558f596f5c-5fxt9 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-94d00-predictor-558f596f5c-5fxt9 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-94d00-predictor-558f596f5c-5fxt9 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-94d00-predictor-85bcffb47b-j4wcf |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-94d00-predictor-85bcffb47b-j4wcf |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1846" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-94d00-predictor-85bcffb47b-j4wcf |
AddedInterface |
Add eth0 [10.132.0.29/23] from ovn-kubernetes | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-315c2-predictor-69cd9f9dfd-h8vcq |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.26:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-315c2-predictor-69cd9f9dfd-h8vcq |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.26:8643/healthz": dial tcp 10.132.0.26:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.28:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.28:8643/healthz": dial tcp 10.133.0.28:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a1525-predictor-7f846d5f79-7nw6m |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a1525-predictor-7f846d5f79-7nw6m |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-a1525-5c67bbd599-28jst |
Killing |
Stopping container ensemble-graph-a1525 | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-a162f-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-a162f-predictor-5955586d98 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-a162f-predictor-5955586d98 |
SuccessfulCreate |
Created pod: success-200-isvc-a162f-predictor-5955586d98-wskzk | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a1525-predictor-596bfc45bb-kpntg |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a1525-predictor-596bfc45bb-kpntg |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-a162f-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-a162f-predictor-56c4c966dc from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a162f-predictor-5955586d98-wskzk |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a162f-predictor-5955586d98-wskzk |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1846" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-a162f-predictor-5955586d98-wskzk |
AddedInterface |
Add eth0 [10.132.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a162f-predictor-56c4c966dc-db74g |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-a162f-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a162f-predictor-5955586d98-wskzk |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-a162f-predictor-56c4c966dc |
SuccessfulCreate |
Created pod: error-404-isvc-a162f-predictor-56c4c966dc-db74g | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a162f-predictor-5955586d98-wskzk |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a162f-predictor-5955586d98-wskzk |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a162f-predictor-5955586d98-wskzk |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a162f-predictor-56c4c966dc-db74g |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-a162f-predictor-56c4c966dc-db74g |
AddedInterface |
Add eth0 [10.133.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a162f-predictor-56c4c966dc-db74g |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1846" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a162f-predictor-56c4c966dc-db74g |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a162f-predictor-56c4c966dc-db74g |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a162f-predictor-56c4c966dc-db74g |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a162f-predictor-56c4c966dc-db74g |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a1525-predictor-596bfc45bb-kpntg |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.28:8643/healthz": dial tcp 10.132.0.28:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a1525-predictor-7f846d5f79-7nw6m |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.29:8643/healthz": dial tcp 10.133.0.29:8643: connect: connection refused | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-94d00-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-94d00-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-a162f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-a162f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-315c2-6dffcdf597-99zwl |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-a1525-5c67bbd599-28jst |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-94d00-predictor-85bcffb47b-j4wcf |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.29:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-94d00-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-94d00-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-94d00-predictor-558f596f5c-5fxt9 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.30:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-94d00 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-94d00 |
InferenceServiceReady |
InferenceService [success-200-isvc-94d00] is Ready | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-a162f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-a162f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-94d00 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-94d00 |
InferenceServiceReady |
InferenceService [error-404-isvc-94d00] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-a162f |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-a162f |
InferenceServiceReady |
InferenceService [success-200-isvc-a162f] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-a162f |
InferenceServiceReady |
InferenceService [error-404-isvc-a162f] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-a162f |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-94d00-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-94d00-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-94d00-868df69ffb-5rp86 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "sequence-graph-94d00-serving-cert" not found | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-94d00 |
InferenceGraphReady |
InferenceGraph [sequence-graph-94d00] is Ready | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-94d00 |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "sequence-graph-94d00": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-94d00 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-94d00-868df69ffb from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-94d00-868df69ffb |
SuccessfulCreate |
Created pod: sequence-graph-94d00-868df69ffb-5rp86 | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-94d00-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-94d00 |
UpdateFailed |
Failed to update status for InferenceGraph "sequence-graph-94d00": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "sequence-graph-94d00": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-94d00-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-94d00-868df69ffb-5rp86 |
Started |
Started container sequence-graph-94d00 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-94d00-868df69ffb-5rp86 |
Created |
Created container: sequence-graph-94d00 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-94d00-868df69ffb-5rp86 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:65dcd869d2c6fd26e4cba78f3f0eef9e2fb99b132c1879d2b886845544ac1220" already present on machine | |
kserve-ci-e2e-test |
multus |
sequence-graph-94d00-868df69ffb-5rp86 |
AddedInterface |
Add eth0 [10.134.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-94d00-868df69ffb-5rp86 |
Killing |
Stopping container sequence-graph-94d00 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-d5455-predictor-788d7ccf85 |
SuccessfulCreate |
Created pod: error-404-isvc-d5455-predictor-788d7ccf85-mskgp | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d5455-predictor-788d7ccf85-mskgp |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-94d00-predictor-85bcffb47b-j4wcf |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-a162f-964d79f84 |
SuccessfulCreate |
Created pod: ensemble-graph-a162f-964d79f84-rgrt2 | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-a162f |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-a162f-964d79f84 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-a162f |
InferenceGraphReady |
InferenceGraph [ensemble-graph-a162f] is Ready | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-a162f |
UpdateFailed |
Failed to update status for InferenceGraph "ensemble-graph-a162f": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "ensemble-graph-a162f": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-a162f |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "ensemble-graph-a162f": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d5455 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-d5455": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-d5455": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d5455 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-d5455": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-d5455-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-d5455-predictor-788d7ccf85 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-a162f-964d79f84-rgrt2 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "ensemble-graph-a162f-serving-cert" not found | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d5455 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-d5455": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-94d00-predictor-85bcffb47b-j4wcf |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d5455-predictor-788d7ccf85-mskgp |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-94d00-predictor-558f596f5c-5fxt9 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d5455-predictor-788d7ccf85-mskgp |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d5455-predictor-788d7ccf85-mskgp |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d5455-predictor-788d7ccf85-mskgp |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d5455-predictor-788d7ccf85-mskgp |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1846" already present on machine | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-a162f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-a162f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d5455-predictor-6776956984-nv48x |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-d5455-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-d5455-predictor-6776956984 |
SuccessfulCreate |
Created pod: success-200-isvc-d5455-predictor-6776956984-nv48x | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-d5455-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-d5455-predictor-6776956984 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-94d00-predictor-558f596f5c-5fxt9 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-d5455-predictor-788d7ccf85-mskgp |
AddedInterface |
Add eth0 [10.133.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d5455 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-d5455": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-d5455": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-a162f-964d79f84-rgrt2 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:65dcd869d2c6fd26e4cba78f3f0eef9e2fb99b132c1879d2b886845544ac1220" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d5455-predictor-6776956984-nv48x |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d5455-predictor-6776956984-nv48x |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-a162f-964d79f84-rgrt2 |
Started |
Started container ensemble-graph-a162f | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d5455-predictor-6776956984-nv48x |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d5455-predictor-6776956984-nv48x |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-94d00-predictor-558f596f5c-5fxt9 |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.30:8643/healthz": dial tcp 10.133.0.30:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d5455-predictor-6776956984-nv48x |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-a162f-964d79f84-rgrt2 |
Created |
Created container: ensemble-graph-a162f | |
kserve-ci-e2e-test |
multus |
ensemble-graph-a162f-964d79f84-rgrt2 |
AddedInterface |
Add eth0 [10.134.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
success-200-isvc-d5455-predictor-6776956984-nv48x |
AddedInterface |
Add eth0 [10.132.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d5455-predictor-6776956984-nv48x |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1846" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-94d00-predictor-85bcffb47b-j4wcf |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.29:8643/healthz": dial tcp 10.132.0.29:8643: connect: connection refused | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-a162f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-a162f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-94d00-868df69ffb-5rp86 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d5455-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-a162f |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-a162f |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d5455-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d5455-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d5455-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-d5455-predictor-6776956984-nv48x |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.31:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d5455 |
InferenceServiceReady |
InferenceService [error-404-isvc-d5455] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d5455 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d5455 |
InferenceServiceReady |
InferenceService [success-200-isvc-d5455] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d5455 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-d5455-58879b467 |
SuccessfulCreate |
Created pod: sequence-graph-d5455-58879b467-qdxfr | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-d5455 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-d5455-58879b467 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-d5455 |
InferenceGraphReady |
InferenceGraph [sequence-graph-d5455] is Ready | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-d5455-58879b467-qdxfr |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "sequence-graph-d5455-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-d5455-58879b467-qdxfr |
Created |
Created container: sequence-graph-d5455 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-d5455-58879b467-qdxfr |
Started |
Started container sequence-graph-d5455 | |
kserve-ci-e2e-test |
multus |
sequence-graph-d5455-58879b467-qdxfr |
AddedInterface |
Add eth0 [10.134.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-d5455-58879b467-qdxfr |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:65dcd869d2c6fd26e4cba78f3f0eef9e2fb99b132c1879d2b886845544ac1220" already present on machine | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d5455-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d5455-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d5455-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d5455-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-d5455 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-d5455 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-a162f-964d79f84-rgrt2 |
Killing |
Stopping container ensemble-graph-a162f | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a162f-predictor-56c4c966dc-db74g |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-8c81a-predictor-f8d8945c6 |
SuccessfulCreate |
Created pod: success-200-isvc-8c81a-predictor-f8d8945c6-v2fbg | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-8c81a |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-8c81a": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8c81a-predictor-756657c85-2pmpb |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1846" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8c81a-predictor-f8d8945c6-v2fbg |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-8c81a-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-8c81a |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-8c81a": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-8c81a |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-8c81a": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-8c81a": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8c81a-predictor-756657c85-2pmpb |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-8c81a |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-8c81a": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-8c81a": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-8c81a-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-8c81a-predictor-756657c85 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a162f-predictor-5955586d98-wskzk |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-8c81a-predictor-756657c85 |
SuccessfulCreate |
Created pod: error-404-isvc-8c81a-predictor-756657c85-2pmpb | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a162f-predictor-56c4c966dc-db74g |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-8c81a-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-8c81a-predictor-f8d8945c6 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8c81a-predictor-756657c85-2pmpb |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8c81a-predictor-756657c85-2pmpb |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8c81a-predictor-756657c85-2pmpb |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a162f-predictor-5955586d98-wskzk |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8c81a-predictor-756657c85-2pmpb |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-8c81a-predictor-756657c85-2pmpb |
AddedInterface |
Add eth0 [10.133.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8c81a-predictor-f8d8945c6-v2fbg |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8c81a-predictor-f8d8945c6-v2fbg |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8c81a-predictor-f8d8945c6-v2fbg |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-8c81a-predictor-f8d8945c6-v2fbg |
AddedInterface |
Add eth0 [10.132.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8c81a-predictor-f8d8945c6-v2fbg |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1846" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8c81a-predictor-f8d8945c6-v2fbg |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8c81a-predictor-f8d8945c6-v2fbg |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a162f-predictor-5955586d98-wskzk |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.30:8643/healthz": dial tcp 10.132.0.30:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-a162f-predictor-5955586d98-wskzk |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.30:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-a162f-predictor-56c4c966dc-db74g |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.31:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a162f-predictor-56c4c966dc-db74g |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.31:8643/healthz": dial tcp 10.133.0.31:8643: connect: connection refused | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8c81a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8c81a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-a162f-964d79f84-rgrt2 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8c81a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8c81a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-8c81a-predictor-756657c85-2pmpb |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.33:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-8c81a-predictor-f8d8945c6-v2fbg |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.32:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-8c81a |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-8c81a |
InferenceServiceReady |
InferenceService [error-404-isvc-8c81a] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-8c81a |
InferenceServiceReady |
InferenceService [success-200-isvc-8c81a] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-8c81a |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d5455-predictor-6776956984-nv48x |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-493ba-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-493ba-predictor-587766b78d from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-d5455-58879b467-qdxfr |
Killing |
Stopping container sequence-graph-d5455 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d5455-predictor-6776956984-nv48x |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-493ba-predictor-6c4f76bbb5 |
SuccessfulCreate |
Created pod: error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d5455-predictor-788d7ccf85-mskgp |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-8c81a-784498d6cc |
SuccessfulCreate |
Created pod: splitter-graph-8c81a-784498d6cc-9qrmg | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-493ba-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-493ba-predictor-6c4f76bbb5 from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-8c81a |
ScalingReplicaSet |
Scaled up replica set splitter-graph-8c81a-784498d6cc from 0 to 1 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8c81a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-8c81a |
InferenceGraphReady |
InferenceGraph [splitter-graph-8c81a] is Ready | |
kserve-ci-e2e-test |
multus |
splitter-graph-8c81a-784498d6cc-9qrmg |
AddedInterface |
Add eth0 [10.134.0.36/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-493ba-predictor-587766b78d-tbfg9 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-493ba-predictor-serving-cert" not found | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8c81a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8c81a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-493ba |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-493ba": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-493ba |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-493ba": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-493ba": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-8c81a-784498d6cc-9qrmg |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:65dcd869d2c6fd26e4cba78f3f0eef9e2fb99b132c1879d2b886845544ac1220" already present on machine | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-8c81a-784498d6cc-9qrmg |
Started |
Started container splitter-graph-8c81a | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-493ba-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-493ba-predictor-587766b78d |
SuccessfulCreate |
Created pod: success-200-isvc-493ba-predictor-587766b78d-tbfg9 | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8c81a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d5455-predictor-788d7ccf85-mskgp |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-8c81a-784498d6cc-9qrmg |
Created |
Created container: splitter-graph-8c81a | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-493ba-predictor-587766b78d-tbfg9 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-493ba-predictor-587766b78d-tbfg9 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-493ba-predictor-587766b78d-tbfg9 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-493ba-predictor-587766b78d-tbfg9 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-493ba-predictor-587766b78d-tbfg9 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv |
AddedInterface |
Add eth0 [10.133.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1846" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-493ba-predictor-587766b78d-tbfg9 |
AddedInterface |
Add eth0 [10.132.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-493ba-predictor-587766b78d-tbfg9 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1846" already present on machine | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-d5455-predictor-788d7ccf85-mskgp |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.32:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d5455-predictor-788d7ccf85-mskgp |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.32:8643/healthz": dial tcp 10.133.0.32:8643: connect: connection refused | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-47ede |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-47ede": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
multus |
error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd |
AddedInterface |
Add eth0 [10.133.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-47ede-predictor-6fd67659b4 |
SuccessfulCreate |
Created pod: success-200-isvc-47ede-predictor-6fd67659b4-bxjfg | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-47ede-predictor-86dbbb4b45 |
SuccessfulCreate |
Created pod: error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8c81a-predictor-f8d8945c6-v2fbg |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8c81a-predictor-f8d8945c6-v2fbg |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-47ede-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-47ede-predictor-6fd67659b4 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-47ede-predictor-6fd67659b4-bxjfg |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-47ede-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-47ede |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-47ede": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-47ede": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-47ede |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-47ede": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-47ede |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-47ede": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-47ede": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-47ede-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-47ede-predictor-86dbbb4b45 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-8c81a-784498d6cc-9qrmg |
Killing |
Stopping container splitter-graph-8c81a | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8c81a-predictor-756657c85-2pmpb |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8c81a-predictor-756657c85-2pmpb |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1846" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-47ede-predictor-6fd67659b4-bxjfg |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1846" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-47ede-predictor-6fd67659b4-bxjfg |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-47ede-predictor-6fd67659b4-bxjfg |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-47ede-predictor-6fd67659b4-bxjfg |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-47ede-predictor-6fd67659b4-bxjfg |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-47ede-predictor-6fd67659b4-bxjfg |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-47ede-predictor-6fd67659b4-bxjfg |
AddedInterface |
Add eth0 [10.132.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8c81a-predictor-756657c85-2pmpb |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.33:8643/healthz": dial tcp 10.133.0.33:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-d5455-58879b467-qdxfr |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-493ba-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-493ba-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-493ba-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-493ba-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-8c81a-784498d6cc-9qrmg |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-493ba-predictor-587766b78d-tbfg9 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.33:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.34:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-493ba |
InferenceServiceReady |
InferenceService [success-200-isvc-493ba] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-493ba |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-493ba |
InferenceServiceReady |
InferenceService [error-404-isvc-493ba] is Ready | |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.35:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-493ba |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-47ede-predictor-6fd67659b4-bxjfg |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.34:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-47ede |
InferenceServiceReady |
InferenceService [error-404-isvc-47ede] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-47ede |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-47ede |
InferenceServiceReady |
InferenceService [success-200-isvc-47ede] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-47ede |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-493ba |
InferenceGraphReady |
InferenceGraph [switch-graph-493ba] is Ready | |
kserve-ci-e2e-test |
kubelet |
switch-graph-493ba-59c7bfbd4c-f2ddc |
Started |
Started container switch-graph-493ba | |
kserve-ci-e2e-test |
multus |
switch-graph-493ba-59c7bfbd4c-f2ddc |
AddedInterface |
Add eth0 [10.134.0.37/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-493ba-59c7bfbd4c-f2ddc |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:65dcd869d2c6fd26e4cba78f3f0eef9e2fb99b132c1879d2b886845544ac1220" already present on machine | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-493ba |
ScalingReplicaSet |
Scaled up replica set switch-graph-493ba-59c7bfbd4c from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-493ba-59c7bfbd4c-f2ddc |
Created |
Created container: switch-graph-493ba | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-493ba-59c7bfbd4c |
SuccessfulCreate |
Created pod: switch-graph-493ba-59c7bfbd4c-f2ddc | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-47ede-d4cf4b666-4btmn |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:65dcd869d2c6fd26e4cba78f3f0eef9e2fb99b132c1879d2b886845544ac1220" already present on machine | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-47ede-d4cf4b666 |
SuccessfulCreate |
Created pod: splitter-graph-47ede-d4cf4b666-4btmn | |
kserve-ci-e2e-test |
multus |
splitter-graph-47ede-d4cf4b666-4btmn |
AddedInterface |
Add eth0 [10.134.0.38/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-47ede |
InferenceGraphReady |
InferenceGraph [splitter-graph-47ede] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-47ede |
ScalingReplicaSet |
Scaled up replica set splitter-graph-47ede-d4cf4b666 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-47ede-d4cf4b666-4btmn |
Created |
Created container: splitter-graph-47ede | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-47ede-d4cf4b666-4btmn |
Started |
Started container splitter-graph-47ede | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-493ba-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-493ba-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-493ba-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-493ba |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-493ba |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-493ba-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-47ede-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-47ede-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-47ede-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-47ede-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-493ba |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-493ba |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-47ede |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-47ede |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-47ede-predictor-6fd67659b4-bxjfg |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-47ede-d4cf4b666-4btmn |
Killing |
Stopping container splitter-graph-47ede | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-47ede-predictor-6fd67659b4-bxjfg |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd |
Killing |
Stopping container kube-rbac-proxy | |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-47ede-d4cf4b666-4btmn |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
switch-graph-493ba-59c7bfbd4c-f2ddc |
Killing |
Stopping container switch-graph-493ba | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-493ba-predictor-587766b78d-tbfg9 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-493ba-predictor-587766b78d-tbfg9 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv |
Killing |
Stopping container kserve-container | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-493ba-59c7bfbd4c-f2ddc |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |