| Time | Namespace | Component | RelatedObject | Reason | Message |
|---|---|---|---|---|---|
kserve-ci-e2e-test |
success-200-isvc-db836-predictor-598457c8f9-bc6fv |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-db836-predictor-598457c8f9-bc6fv to ip-10-0-137-180.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-52e43-predictor-674cd6cc9-prtkw |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-52e43-predictor-674cd6cc9-prtkw to ip-10-0-137-180.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-6460c-7c977b8767-wcnvq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-6460c-7c977b8767-wcnvq to ip-10-0-138-165.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-bbf0f-c98544745-zdsfs |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-bbf0f-c98544745-zdsfs to ip-10-0-138-165.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-1b15a-predictor-84b9dd6865-v2nnf |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-1b15a-predictor-84b9dd6865-v2nnf to ip-10-0-137-180.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-3c336-predictor-54665d447b-s2vc2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-3c336-predictor-54665d447b-s2vc2 to ip-10-0-137-180.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-db836-5758686fdc-568g5 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-db836-5758686fdc-568g5 to ip-10-0-138-165.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww to ip-10-0-140-168.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-ce016-7b6cbf87f8-94t72 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-ce016-7b6cbf87f8-94t72 to ip-10-0-138-165.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-4517b-predictor-677799f794-kj8t9 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-4517b-predictor-677799f794-kj8t9 to ip-10-0-137-180.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-bbcac-77d4868ccd-zlxf7 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-bbcac-77d4868ccd-zlxf7 to ip-10-0-138-165.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-1b15a-5bf689b779-zll7z |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-1b15a-5bf689b779-zll7z to ip-10-0-137-180.ec2.internal | ||
kserve-ci-e2e-test |
isvc-xgboost-graph-predictor-555fc69d56-2p587 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587 to ip-10-0-140-168.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-ce016-predictor-6ffdb68b79-hq49w |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-ce016-predictor-6ffdb68b79-hq49w to ip-10-0-137-180.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-bbcac-predictor-7874fb45b7-bmwwk |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-bbcac-predictor-7874fb45b7-bmwwk to ip-10-0-137-180.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-1f194-54bc64db78-46rl4 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-1f194-54bc64db78-46rl4 to ip-10-0-138-165.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-1f194-predictor-6dd685749f-bq5w2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-1f194-predictor-6dd685749f-bq5w2 to ip-10-0-137-180.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-6460c-predictor-5d5d94dfb6-btq8d |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-6460c-predictor-5d5d94dfb6-btq8d to ip-10-0-137-180.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-6460c-predictor-5774675f9-x8h72 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-6460c-predictor-5774675f9-x8h72 to ip-10-0-137-180.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-bbcac-predictor-5ff4fd5cc4-wtk7h |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-bbcac-predictor-5ff4fd5cc4-wtk7h to ip-10-0-137-180.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-52e43-65c965b44d-tmkgc |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-52e43-65c965b44d-tmkgc to ip-10-0-138-165.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-bbf0f-predictor-6d7cf8c5c-bc4gn |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-bbf0f-predictor-6d7cf8c5c-bc4gn to ip-10-0-137-180.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-1f194-predictor-6c5d4fc46b-kk5sz |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-1f194-predictor-6c5d4fc46b-kk5sz to ip-10-0-137-180.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-3c336-67f978898d-48sxc |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-3c336-67f978898d-48sxc to ip-10-0-138-165.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-52e43-predictor-79768fc67d-xk27k |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-52e43-predictor-79768fc67d-xk27k to ip-10-0-137-180.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-1b15a-predictor-5c76b7cdd8-56vrx |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-1b15a-predictor-5c76b7cdd8-56vrx to ip-10-0-137-180.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-1-predictor-875bcd459-pbj9r |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-pbj9r to ip-10-0-138-165.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-4517b-7b778f989-h6fmq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-4517b-7b778f989-h6fmq to ip-10-0-138-165.ec2.internal | ||
kserve-ci-e2e-test |
model-chainer-54fd6d96b6-gg5gt |
Scheduled |
Successfully assigned kserve-ci-e2e-test/model-chainer-54fd6d96b6-gg5gt to ip-10-0-138-165.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-db836-predictor-6996c59df8-gc6q8 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-db836-predictor-6996c59df8-gc6q8 to ip-10-0-137-180.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-ce016-predictor-84dbff7cdb-rdb8n |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-ce016-predictor-84dbff7cdb-rdb8n to ip-10-0-137-180.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-3c336-predictor-547bb6bc8f-96z6b |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-3c336-predictor-547bb6bc8f-96z6b to ip-10-0-137-180.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-bbf0f-predictor-85ff6cdb4-jzkmn |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-bbf0f-predictor-85ff6cdb4-jzkmn to ip-10-0-137-180.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-4517b-predictor-64b9b9f5bc-9pcrm |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-4517b-predictor-64b9b9f5bc-9pcrm to ip-10-0-137-180.ec2.internal | ||
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-2-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-2-predictor-6c8478f9b5 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-2-predictor-6c8478f9b5 |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-1-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-1-predictor-875bcd459 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
success-200-isvc-1b15a-predictor-84b9dd6865-v2nnf |
AddedInterface |
Add eth0 [10.132.0.23/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b15a-predictor-84b9dd6865-v2nnf |
Pulling |
Pulling image "quay.io/opendatahub/success-200-isvc:odh-pr-1690" | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-1b15a-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-1b15a-predictor-84b9dd6865 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-1-predictor-875bcd459-pbj9r |
AddedInterface |
Add eth0 [10.134.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-1b15a-predictor-5c76b7cdd8 |
SuccessfulCreate |
Created pod: error-404-isvc-1b15a-predictor-5c76b7cdd8-56vrx | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-1b15a-predictor-84b9dd6865 |
SuccessfulCreate |
Created pod: success-200-isvc-1b15a-predictor-84b9dd6865-v2nnf | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b15a-predictor-5c76b7cdd8-56vrx |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-1b15a-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-1b15a-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-1b15a-predictor-5c76b7cdd8 from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "isvc-sklearn-graph-1": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-875bcd459-pbj9r |
Pulling |
Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:067fec2d500e8a88ee10f3be03a35a3f15d715d25b8b6a966457243c4e7a45a0" | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-xgboost-graph-predictor-555fc69d56 |
SuccessfulCreate |
Created pod: isvc-xgboost-graph-predictor-555fc69d56-2p587 | |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
UpdateFailed |
Failed to update status for InferenceService "isvc-sklearn-graph-1": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "isvc-sklearn-graph-1": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-1-predictor-875bcd459 |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-1-predictor-875bcd459-pbj9r | |
kserve-ci-e2e-test |
multus |
isvc-xgboost-graph-predictor-555fc69d56-2p587 |
AddedInterface |
Add eth0 [10.133.0.22/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
deployment-controller |
isvc-xgboost-graph-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-xgboost-graph-predictor-555fc69d56 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
error-404-isvc-1b15a-predictor-5c76b7cdd8-56vrx |
AddedInterface |
Add eth0 [10.132.0.24/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2p587 |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:067fec2d500e8a88ee10f3be03a35a3f15d715d25b8b6a966457243c4e7a45a0" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "isvc-sklearn-graph-2-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2p587 |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:067fec2d500e8a88ee10f3be03a35a3f15d715d25b8b6a966457243c4e7a45a0" already present on machine | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww |
AddedInterface |
Add eth0 [10.133.0.23/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2p587 |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b15a-predictor-5c76b7cdd8-56vrx |
Pulling |
Pulling image "quay.io/opendatahub/error-404-isvc:odh-pr-1690" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-875bcd459-pbj9r |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-875bcd459-pbj9r |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-875bcd459-pbj9r |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:067fec2d500e8a88ee10f3be03a35a3f15d715d25b8b6a966457243c4e7a45a0" in 4.163s (4.163s including waiting). Image size: 301878744 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2p587 |
Pulling |
Pulling image "kserve/xgbserver:latest" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1690" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-875bcd459-pbj9r |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1690" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b15a-predictor-5c76b7cdd8-56vrx |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b15a-predictor-84b9dd6865-v2nnf |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b15a-predictor-84b9dd6865-v2nnf |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b15a-predictor-84b9dd6865-v2nnf |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b15a-predictor-5c76b7cdd8-56vrx |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b15a-predictor-5c76b7cdd8-56vrx |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b15a-predictor-5c76b7cdd8-56vrx |
Pulled |
Successfully pulled image "quay.io/opendatahub/error-404-isvc:odh-pr-1690" in 17.641s (17.641s including waiting). Image size: 1334626409 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b15a-predictor-84b9dd6865-v2nnf |
Pulled |
Successfully pulled image "quay.io/opendatahub/success-200-isvc:odh-pr-1690" in 18.529s (18.529s including waiting). Image size: 1335612528 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1690" in 15.102s (15.102s including waiting). Image size: 1555871157 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-875bcd459-pbj9r |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b15a-predictor-5c76b7cdd8-56vrx |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b15a-predictor-84b9dd6865-v2nnf |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-875bcd459-pbj9r |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1690" in 12.705s (12.705s including waiting). Image size: 1555871157 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-875bcd459-pbj9r |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b15a-predictor-84b9dd6865-v2nnf |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.977s (2.977s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b15a-predictor-84b9dd6865-v2nnf |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-875bcd459-pbj9r |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b15a-predictor-5c76b7cdd8-56vrx |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.991s (2.991s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b15a-predictor-5c76b7cdd8-56vrx |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-875bcd459-pbj9r |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.187s (2.187s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-875bcd459-pbj9r |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-875bcd459-pbj9r |
Started |
Started container kube-rbac-proxy | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1b15a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1b15a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1b15a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1b15a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2p587 |
Pulled |
Successfully pulled image "kserve/xgbserver:latest" in 25.646s (25.646s including waiting). Image size: 1305293052 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2p587 |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2p587 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2p587 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2p587 |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 1.937s (1.937s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2p587 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 11.556s (11.556s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2p587 |
Started |
Started container kube-rbac-proxy | |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b15a-predictor-5c76b7cdd8-56vrx |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.24:8080: connect: connection refused |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b15a-predictor-84b9dd6865-v2nnf |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.23:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-1b15a |
InferenceServiceReady |
InferenceService [error-404-isvc-1b15a] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-1b15a |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-1b15a |
InferenceServiceReady |
InferenceService [success-200-isvc-1b15a] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-1b15a |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x8) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.23:8080: connect: connection refused |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1b15a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
switch-graph-1b15a-5bf689b779-zll7z |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-1b15a-serving-cert" not found | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1b15a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
InferenceServiceReady |
InferenceService [isvc-xgboost-graph] is Ready | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1b15a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-1b15a |
ScalingReplicaSet |
Scaled up replica set switch-graph-1b15a-5bf689b779 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-1b15a-5bf689b779 |
SuccessfulCreate |
Created pod: switch-graph-1b15a-5bf689b779-zll7z | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-1b15a |
InferenceGraphReady |
InferenceGraph [switch-graph-1b15a] is Ready | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1b15a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
multus |
switch-graph-1b15a-5bf689b779-zll7z |
AddedInterface |
Add eth0 [10.132.0.25/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-1b15a-5bf689b779-zll7z |
Pulling |
Pulling image "quay.io/opendatahub/kserve-router@sha256:c0785f60b462f861cfab04ad440fa6adb06dbacec8c1d6b98ebf808ae746d9cf" | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x9) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-875bcd459-pbj9r |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.30:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
switch-graph-1b15a-5bf689b779-zll7z |
Started |
Started container switch-graph-1b15a | |
kserve-ci-e2e-test |
kubelet |
switch-graph-1b15a-5bf689b779-zll7z |
Created |
Created container: switch-graph-1b15a | |
kserve-ci-e2e-test |
kubelet |
switch-graph-1b15a-5bf689b779-zll7z |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-router@sha256:c0785f60b462f861cfab04ad440fa6adb06dbacec8c1d6b98ebf808ae746d9cf" in 2.617s (2.617s including waiting). Image size: 218070140 bytes. | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-1] is Ready | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-2] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-3c336 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-3c336": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b15a-predictor-5c76b7cdd8-56vrx |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3c336-predictor-547bb6bc8f-96z6b |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-3c336-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b15a-predictor-84b9dd6865-v2nnf |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b15a-predictor-84b9dd6865-v2nnf |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-3c336-predictor-547bb6bc8f |
SuccessfulCreate |
Created pod: success-200-isvc-3c336-predictor-547bb6bc8f-96z6b | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-3c336-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-3c336-predictor-547bb6bc8f from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-3c336 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-3c336": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-3c336": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-3c336-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-3c336-predictor-54665d447b from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-3c336-predictor-54665d447b |
SuccessfulCreate |
Created pod: error-404-isvc-3c336-predictor-54665d447b-s2vc2 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-3c336 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-3c336": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-3c336": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-3c336 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-3c336": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
switch-graph-1b15a-5bf689b779-zll7z |
Killing |
Stopping container switch-graph-1b15a | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b15a-predictor-5c76b7cdd8-56vrx |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3c336-predictor-54665d447b-s2vc2 |
Started |
Started container kserve-container | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b15a-predictor-84b9dd6865-v2nnf |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.23:8643/healthz": dial tcp 10.132.0.23:8643: connect: connection refused | |
kserve-ci-e2e-test |
multus |
success-200-isvc-3c336-predictor-547bb6bc8f-96z6b |
AddedInterface |
Add eth0 [10.132.0.26/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3c336-predictor-547bb6bc8f-96z6b |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1690" already present on machine | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3c336-predictor-547bb6bc8f-96z6b |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3c336-predictor-547bb6bc8f-96z6b |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3c336-predictor-54665d447b-s2vc2 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3c336-predictor-54665d447b-s2vc2 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3c336-predictor-547bb6bc8f-96z6b |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3c336-predictor-547bb6bc8f-96z6b |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-3c336-predictor-54665d447b-s2vc2 |
AddedInterface |
Add eth0 [10.132.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3c336-predictor-54665d447b-s2vc2 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1690" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3c336-predictor-54665d447b-s2vc2 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3c336-predictor-547bb6bc8f-96z6b |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3c336-predictor-54665d447b-s2vc2 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
replicaset-controller |
model-chainer-54fd6d96b6 |
SuccessfulCreate |
Created pod: model-chainer-54fd6d96b6-gg5gt | |
kserve-ci-e2e-test |
multus |
model-chainer-54fd6d96b6-gg5gt |
AddedInterface |
Add eth0 [10.134.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
model-chainer-54fd6d96b6-gg5gt |
Pulling |
Pulling image "quay.io/opendatahub/kserve-router@sha256:c0785f60b462f861cfab04ad440fa6adb06dbacec8c1d6b98ebf808ae746d9cf" | |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
model-chainer |
InferenceGraphReady |
InferenceGraph [model-chainer] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
model-chainer |
ScalingReplicaSet |
Scaled up replica set model-chainer-54fd6d96b6 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
model-chainer-54fd6d96b6-gg5gt |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-router@sha256:c0785f60b462f861cfab04ad440fa6adb06dbacec8c1d6b98ebf808ae746d9cf" in 2.593s (2.593s including waiting). Image size: 218070140 bytes. | |
kserve-ci-e2e-test |
kubelet |
model-chainer-54fd6d96b6-gg5gt |
Created |
Created container: model-chainer | |
kserve-ci-e2e-test |
kubelet |
model-chainer-54fd6d96b6-gg5gt |
Started |
Started container model-chainer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
model-chainer-54fd6d96b6-gg5gt |
Killing |
Stopping container model-chainer | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-ce016-predictor-6ffdb68b79 |
SuccessfulCreate |
Created pod: success-200-isvc-ce016-predictor-6ffdb68b79-hq49w | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-ce016-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-ce016-predictor-6ffdb68b79 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ce016-predictor-6ffdb68b79-hq49w |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-ce016-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-875bcd459-pbj9r |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-875bcd459-pbj9r |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-ce016-predictor-84dbff7cdb |
SuccessfulCreate |
Created pod: error-404-isvc-ce016-predictor-84dbff7cdb-rdb8n | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2p587 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww |
Killing |
Stopping container kube-rbac-proxy | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-1b15a-5bf689b779-zll7z |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2p587 |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.22:8643/healthz": dial tcp 10.133.0.22:8643: connect: connection refused | |
| (x8) | kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2p587 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.22:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-2p587 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-ce016-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-ce016-predictor-84dbff7cdb from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ce016-predictor-6ffdb68b79-hq49w |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ce016-predictor-6ffdb68b79-hq49w |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ce016-predictor-6ffdb68b79-hq49w |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1690" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-ce016-predictor-6ffdb68b79-hq49w |
AddedInterface |
Add eth0 [10.132.0.28/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ce016-predictor-6ffdb68b79-hq49w |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-875bcd459-pbj9r |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.30:8643/healthz": dial tcp 10.134.0.30:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ce016-predictor-6ffdb68b79-hq49w |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-ce016-predictor-84dbff7cdb-rdb8n |
AddedInterface |
Add eth0 [10.132.0.29/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ce016-predictor-84dbff7cdb-rdb8n |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1690" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ce016-predictor-84dbff7cdb-rdb8n |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ce016-predictor-84dbff7cdb-rdb8n |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ce016-predictor-84dbff7cdb-rdb8n |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ce016-predictor-84dbff7cdb-rdb8n |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ce016-predictor-84dbff7cdb-rdb8n |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ce016-predictor-6ffdb68b79-hq49w |
Started |
Started container kube-rbac-proxy | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-3c336-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-3c336-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-3c336-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-3c336-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.23:8643/healthz": context deadline exceeded | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-ce016-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-ce016-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-3c336 |
InferenceServiceReady |
InferenceService [error-404-isvc-3c336] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-3c336 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
kubelet |
model-chainer-54fd6d96b6-gg5gt |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-3c336 |
InferenceServiceReady |
InferenceService [success-200-isvc-3c336] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-3c336 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-ce016-predictor-6ffdb68b79-hq49w |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.28:8080: connect: connection refused |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-3c336 |
InferenceGraphReady |
InferenceGraph [switch-graph-3c336] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-3c336 |
ScalingReplicaSet |
Scaled up replica set switch-graph-3c336-67f978898d from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-3c336-67f978898d |
SuccessfulCreate |
Created pod: switch-graph-3c336-67f978898d-48sxc | |
kserve-ci-e2e-test |
kubelet |
switch-graph-3c336-67f978898d-48sxc |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-3c336-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
switch-graph-3c336-67f978898d-48sxc |
Started |
Started container switch-graph-3c336 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-3c336-67f978898d-48sxc |
Created |
Created container: switch-graph-3c336 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-3c336-67f978898d-48sxc |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:c0785f60b462f861cfab04ad440fa6adb06dbacec8c1d6b98ebf808ae746d9cf" already present on machine | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-ce016-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
multus |
switch-graph-3c336-67f978898d-48sxc |
AddedInterface |
Add eth0 [10.134.0.32/23] from ovn-kubernetes | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-ce016-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-ce016 |
InferenceServiceReady |
InferenceService [error-404-isvc-ce016] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-ce016 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-3c336-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-ce016 |
InferenceServiceReady |
InferenceService [success-200-isvc-ce016] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-ce016 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-3c336-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
multus |
sequence-graph-ce016-7b6cbf87f8-94t72 |
AddedInterface |
Add eth0 [10.134.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-ce016-7b6cbf87f8-94t72 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:c0785f60b462f861cfab04ad440fa6adb06dbacec8c1d6b98ebf808ae746d9cf" already present on machine | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-ce016-7b6cbf87f8-94t72 |
Created |
Created container: sequence-graph-ce016 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-ce016-7b6cbf87f8-94t72 |
Started |
Started container sequence-graph-ce016 | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-ce016-7b6cbf87f8 |
SuccessfulCreate |
Created pod: sequence-graph-ce016-7b6cbf87f8-94t72 | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-ce016 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-ce016-7b6cbf87f8 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-ce016 |
InferenceGraphReady |
InferenceGraph [sequence-graph-ce016] is Ready | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-3c336-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-3c336-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-ce016 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-ce016 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-ce016-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-ce016-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-3c336 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-3c336 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-ce016 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-ce016 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-ce016-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-ce016-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3c336-predictor-54665d447b-s2vc2 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3c336-predictor-54665d447b-s2vc2 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-52e43-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-52e43-predictor-79768fc67d from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-52e43 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-52e43": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-52e43 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-52e43": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-52e43": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
switch-graph-3c336-67f978898d-48sxc |
Killing |
Stopping container switch-graph-3c336 | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-52e43-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-52e43-predictor-674cd6cc9 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-52e43-predictor-79768fc67d |
SuccessfulCreate |
Created pod: error-404-isvc-52e43-predictor-79768fc67d-xk27k | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-52e43-predictor-674cd6cc9 |
SuccessfulCreate |
Created pod: success-200-isvc-52e43-predictor-674cd6cc9-prtkw | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-52e43 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-52e43": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-52e43 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-52e43": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-52e43": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3c336-predictor-547bb6bc8f-96z6b |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3c336-predictor-547bb6bc8f-96z6b |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-52e43-predictor-674cd6cc9-prtkw |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-52e43-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-52e43-predictor-674cd6cc9-prtkw |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-52e43-predictor-79768fc67d-xk27k |
AddedInterface |
Add eth0 [10.132.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-52e43-predictor-79768fc67d-xk27k |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-52e43-predictor-79768fc67d-xk27k |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-52e43-predictor-79768fc67d-xk27k |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-52e43-predictor-79768fc67d-xk27k |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1690" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-52e43-predictor-674cd6cc9-prtkw |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-52e43-predictor-674cd6cc9-prtkw |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1690" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-52e43-predictor-674cd6cc9-prtkw |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-52e43-predictor-79768fc67d-xk27k |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-52e43-predictor-674cd6cc9-prtkw |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-52e43-predictor-79768fc67d-xk27k |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-52e43-predictor-674cd6cc9-prtkw |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
multus |
success-200-isvc-52e43-predictor-674cd6cc9-prtkw |
AddedInterface |
Add eth0 [10.132.0.30/23] from ovn-kubernetes | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-3c336-predictor-54665d447b-s2vc2 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.27:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3c336-predictor-54665d447b-s2vc2 |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.27:8643/healthz": dial tcp 10.132.0.27:8643: connect: connection refused | |
| (x8) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-3c336-predictor-547bb6bc8f-96z6b |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.26:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3c336-predictor-547bb6bc8f-96z6b |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.26:8643/healthz": dial tcp 10.132.0.26:8643: connect: connection refused | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-52e43-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-52e43-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-52e43-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-52e43-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-1f194 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-1f194": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-1f194 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-1f194": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-1f194": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-1f194-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-1f194-predictor-6c5d4fc46b from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-1f194-predictor-6c5d4fc46b |
SuccessfulCreate |
Created pod: error-404-isvc-1f194-predictor-6c5d4fc46b-kk5sz | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1f194-predictor-6c5d4fc46b-kk5sz |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-1f194-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ce016-predictor-84dbff7cdb-rdb8n |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ce016-predictor-6ffdb68b79-hq49w |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ce016-predictor-6ffdb68b79-hq49w |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ce016-predictor-84dbff7cdb-rdb8n |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-ce016-7b6cbf87f8-94t72 |
Killing |
Stopping container sequence-graph-ce016 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-1f194-predictor-6dd685749f |
SuccessfulCreate |
Created pod: success-200-isvc-1f194-predictor-6dd685749f-bq5w2 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1f194-predictor-6dd685749f-bq5w2 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-1f194-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-1f194-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-1f194-predictor-6dd685749f from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1f194-predictor-6dd685749f-bq5w2 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1f194-predictor-6c5d4fc46b-kk5sz |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1f194-predictor-6dd685749f-bq5w2 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1f194-predictor-6c5d4fc46b-kk5sz |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1f194-predictor-6c5d4fc46b-kk5sz |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-1f194-predictor-6c5d4fc46b-kk5sz |
AddedInterface |
Add eth0 [10.132.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1f194-predictor-6dd685749f-bq5w2 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1690" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1f194-predictor-6dd685749f-bq5w2 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1f194-predictor-6dd685749f-bq5w2 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1f194-predictor-6dd685749f-bq5w2 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1f194-predictor-6c5d4fc46b-kk5sz |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1690" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1f194-predictor-6c5d4fc46b-kk5sz |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1f194-predictor-6c5d4fc46b-kk5sz |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
success-200-isvc-1f194-predictor-6dd685749f-bq5w2 |
AddedInterface |
Add eth0 [10.132.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ce016-predictor-84dbff7cdb-rdb8n |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.29:8643/healthz": dial tcp 10.132.0.29:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-ce016-predictor-84dbff7cdb-rdb8n |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.29:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-3c336-67f978898d-48sxc |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-52e43-predictor-674cd6cc9-prtkw |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.30:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-ce016-7b6cbf87f8-94t72 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-52e43-predictor-79768fc67d-xk27k |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.31:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-52e43 |
InferenceServiceReady |
InferenceService [success-200-isvc-52e43] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-52e43 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-52e43 |
InferenceServiceReady |
InferenceService [error-404-isvc-52e43] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-52e43 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-52e43-65c965b44d |
SuccessfulCreate |
Created pod: ensemble-graph-52e43-65c965b44d-tmkgc | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-52e43-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-52e43-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-52e43-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
multus |
ensemble-graph-52e43-65c965b44d-tmkgc |
AddedInterface |
Add eth0 [10.134.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-52e43-65c965b44d-tmkgc |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:c0785f60b462f861cfab04ad440fa6adb06dbacec8c1d6b98ebf808ae746d9cf" already present on machine | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-52e43-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-52e43 |
InferenceGraphReady |
InferenceGraph [ensemble-graph-52e43] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-52e43 |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-52e43-65c965b44d from 0 to 1 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1f194-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-52e43-65c965b44d-tmkgc |
Started |
Started container ensemble-graph-52e43 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-52e43-65c965b44d-tmkgc |
Created |
Created container: ensemble-graph-52e43 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1f194-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1f194-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1f194-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-1f194-predictor-6dd685749f-bq5w2 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.32:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-1f194-predictor-6c5d4fc46b-kk5sz |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.33:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-bbcac |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-bbcac": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-bbcac": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-52e43-65c965b44d-tmkgc |
Killing |
Stopping container ensemble-graph-52e43 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-bbcac |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-bbcac": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-52e43-predictor-674cd6cc9-prtkw |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-bbcac-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-bbcac-predictor-7874fb45b7 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-52e43-predictor-674cd6cc9-prtkw |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-bbcac-predictor-5ff4fd5cc4 |
SuccessfulCreate |
Created pod: error-404-isvc-bbcac-predictor-5ff4fd5cc4-wtk7h | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-bbcac-predictor-7874fb45b7 |
SuccessfulCreate |
Created pod: success-200-isvc-bbcac-predictor-7874fb45b7-bmwwk | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-52e43-predictor-79768fc67d-xk27k |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-bbcac-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-bbcac-predictor-5ff4fd5cc4 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-52e43-predictor-79768fc67d-xk27k |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-bbcac-predictor-5ff4fd5cc4-wtk7h |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1690" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-bbcac-predictor-7874fb45b7-bmwwk |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-bbcac-predictor-7874fb45b7-bmwwk |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-bbcac-predictor-7874fb45b7-bmwwk |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1690" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-bbcac-predictor-7874fb45b7-bmwwk |
AddedInterface |
Add eth0 [10.132.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-bbcac-predictor-7874fb45b7-bmwwk |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-bbcac-predictor-7874fb45b7-bmwwk |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-bbcac-predictor-5ff4fd5cc4-wtk7h |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-bbcac-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
multus |
error-404-isvc-bbcac-predictor-5ff4fd5cc4-wtk7h |
AddedInterface |
Add eth0 [10.132.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-bbcac-predictor-7874fb45b7-bmwwk |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-bbcac-predictor-5ff4fd5cc4-wtk7h |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-bbcac-predictor-5ff4fd5cc4-wtk7h |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-bbcac-predictor-5ff4fd5cc4-wtk7h |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-bbcac-predictor-5ff4fd5cc4-wtk7h |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-bbcac-predictor-5ff4fd5cc4-wtk7h |
Started |
Started container kserve-container | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-1f194 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-1f194 |
InferenceServiceReady |
InferenceService [success-200-isvc-1f194] is Ready | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-52e43-predictor-79768fc67d-xk27k |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.31:8643/healthz": dial tcp 10.132.0.31:8643: connect: connection refused | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-1f194 |
InferenceServiceReady |
InferenceService [error-404-isvc-1f194] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-1f194 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-1f194 |
InferenceGraphReady |
InferenceGraph [sequence-graph-1f194] is Ready | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-1f194-54bc64db78-46rl4 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "sequence-graph-1f194-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-1f194-54bc64db78 |
SuccessfulCreate |
Created pod: sequence-graph-1f194-54bc64db78-46rl4 | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-1f194 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-1f194-54bc64db78 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-1f194-54bc64db78-46rl4 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:c0785f60b462f861cfab04ad440fa6adb06dbacec8c1d6b98ebf808ae746d9cf" already present on machine | |
kserve-ci-e2e-test |
multus |
sequence-graph-1f194-54bc64db78-46rl4 |
AddedInterface |
Add eth0 [10.134.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-1f194-54bc64db78-46rl4 |
Created |
Created container: sequence-graph-1f194 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-1f194-54bc64db78-46rl4 |
Started |
Started container sequence-graph-1f194 | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1f194-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1f194-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1f194-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1f194-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1f194-predictor-6dd685749f-bq5w2 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-6460c |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-6460c": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1f194-predictor-6c5d4fc46b-kk5sz |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-6460c-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-6460c-predictor-5d5d94dfb6 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-6460c-predictor-5d5d94dfb6 |
SuccessfulCreate |
Created pod: error-404-isvc-6460c-predictor-5d5d94dfb6-btq8d | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6460c-predictor-5774675f9-x8h72 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-6460c-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-6460c-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-6460c-predictor-5774675f9 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1f194-predictor-6dd685749f-bq5w2 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-1f194-54bc64db78-46rl4 |
Killing |
Stopping container sequence-graph-1f194 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1f194-predictor-6c5d4fc46b-kk5sz |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-6460c |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-6460c": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-6460c": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-6460c-predictor-5774675f9 |
SuccessfulCreate |
Created pod: success-200-isvc-6460c-predictor-5774675f9-x8h72 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6460c-predictor-5774675f9-x8h72 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6460c-predictor-5774675f9-x8h72 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1690" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6460c-predictor-5d5d94dfb6-btq8d |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6460c-predictor-5d5d94dfb6-btq8d |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6460c-predictor-5774675f9-x8h72 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6460c-predictor-5774675f9-x8h72 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6460c-predictor-5774675f9-x8h72 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6460c-predictor-5774675f9-x8h72 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-6460c-predictor-5d5d94dfb6-btq8d |
AddedInterface |
Add eth0 [10.132.0.37/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
success-200-isvc-6460c-predictor-5774675f9-x8h72 |
AddedInterface |
Add eth0 [10.132.0.36/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6460c-predictor-5d5d94dfb6-btq8d |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6460c-predictor-5d5d94dfb6-btq8d |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6460c-predictor-5d5d94dfb6-btq8d |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6460c-predictor-5d5d94dfb6-btq8d |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1690" already present on machine | |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-52e43-65c965b44d-tmkgc |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1f194-predictor-6c5d4fc46b-kk5sz |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.33:8643/healthz": dial tcp 10.132.0.33:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1f194-predictor-6dd685749f-bq5w2 |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.32:8643/healthz": dial tcp 10.132.0.32:8643: connect: connection refused | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-bbcac-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-bbcac-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-bbcac-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-bbcac-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-bbcac-predictor-5ff4fd5cc4-wtk7h |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.35:8080: connect: connection refused |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-bbcac |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-bbcac |
InferenceServiceReady |
InferenceService [success-200-isvc-bbcac] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-bbcac |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-bbcac |
InferenceServiceReady |
InferenceService [error-404-isvc-bbcac] is Ready | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-1f194-54bc64db78-46rl4 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-6460c-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-6460c-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-6460c-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-6460c-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-bbcac |
InferenceGraphReady |
InferenceGraph [ensemble-graph-bbcac] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-bbcac |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-bbcac-77d4868ccd from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-bbcac-77d4868ccd |
SuccessfulCreate |
Created pod: ensemble-graph-bbcac-77d4868ccd-zlxf7 | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-bbcac |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "ensemble-graph-bbcac": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-bbcac |
UpdateFailed |
Failed to update status for InferenceGraph "ensemble-graph-bbcac": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "ensemble-graph-bbcac": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-bbcac-77d4868ccd-zlxf7 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:c0785f60b462f861cfab04ad440fa6adb06dbacec8c1d6b98ebf808ae746d9cf" already present on machine | |
kserve-ci-e2e-test |
multus |
ensemble-graph-bbcac-77d4868ccd-zlxf7 |
AddedInterface |
Add eth0 [10.134.0.36/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-bbcac-77d4868ccd-zlxf7 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "ensemble-graph-bbcac-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-bbcac-77d4868ccd-zlxf7 |
Started |
Started container ensemble-graph-bbcac | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-bbcac-77d4868ccd-zlxf7 |
Created |
Created container: ensemble-graph-bbcac | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-6460c-predictor-5774675f9-x8h72 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.36:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-6460c |
InferenceServiceReady |
InferenceService [error-404-isvc-6460c] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-6460c |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-bbcac-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-6460c |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-bbcac-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-bbcac-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-6460c |
InferenceServiceReady |
InferenceService [success-200-isvc-6460c] is Ready | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-bbcac-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
sequence-graph-6460c-7c977b8767-wcnvq |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "sequence-graph-6460c-serving-cert" not found | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-6460c |
InferenceGraphReady |
InferenceGraph [sequence-graph-6460c] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-6460c |
ScalingReplicaSet |
Scaled up replica set sequence-graph-6460c-7c977b8767 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-6460c-7c977b8767 |
SuccessfulCreate |
Created pod: sequence-graph-6460c-7c977b8767-wcnvq | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-6460c |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "sequence-graph-6460c": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-6460c |
UpdateFailed |
Failed to update status for InferenceGraph "sequence-graph-6460c": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "sequence-graph-6460c": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-6460c-7c977b8767-wcnvq |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:c0785f60b462f861cfab04ad440fa6adb06dbacec8c1d6b98ebf808ae746d9cf" already present on machine | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-6460c-7c977b8767-wcnvq |
Created |
Created container: sequence-graph-6460c | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-6460c-7c977b8767-wcnvq |
Started |
Started container sequence-graph-6460c | |
kserve-ci-e2e-test |
multus |
sequence-graph-6460c-7c977b8767-wcnvq |
AddedInterface |
Add eth0 [10.134.0.37/23] from ovn-kubernetes | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-6460c-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-6460c-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-6460c-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-6460c-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-bbcac |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-bbcac |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-6460c |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-6460c |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-bbcac-predictor-5ff4fd5cc4-wtk7h |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-bbcac-predictor-7874fb45b7-bmwwk |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-bbcac-predictor-5ff4fd5cc4-wtk7h |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-4517b-predictor-64b9b9f5bc |
SuccessfulCreate |
Created pod: success-200-isvc-4517b-predictor-64b9b9f5bc-9pcrm | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-4517b-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-4517b-predictor-64b9b9f5bc from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-bbcac-77d4868ccd-zlxf7 |
Killing |
Stopping container ensemble-graph-bbcac | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-4517b-predictor-677799f794 |
SuccessfulCreate |
Created pod: error-404-isvc-4517b-predictor-677799f794-kj8t9 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-4517b |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-4517b": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-4517b": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-bbcac-predictor-7874fb45b7-bmwwk |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-4517b-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-4517b-predictor-677799f794 from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-4517b |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-4517b": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-4517b": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-4517b |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-4517b": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4517b-predictor-677799f794-kj8t9 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-4517b-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-4517b |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-4517b": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4517b-predictor-64b9b9f5bc-9pcrm |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4517b-predictor-677799f794-kj8t9 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4517b-predictor-64b9b9f5bc-9pcrm |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4517b-predictor-677799f794-kj8t9 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4517b-predictor-677799f794-kj8t9 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4517b-predictor-677799f794-kj8t9 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4517b-predictor-677799f794-kj8t9 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4517b-predictor-64b9b9f5bc-9pcrm |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-4517b-predictor-64b9b9f5bc-9pcrm |
AddedInterface |
Add eth0 [10.132.0.38/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4517b-predictor-64b9b9f5bc-9pcrm |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1690" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4517b-predictor-64b9b9f5bc-9pcrm |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4517b-predictor-64b9b9f5bc-9pcrm |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-4517b-predictor-677799f794-kj8t9 |
AddedInterface |
Add eth0 [10.132.0.39/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4517b-predictor-677799f794-kj8t9 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1690" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-bbcac-predictor-7874fb45b7-bmwwk |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.34:8643/healthz": dial tcp 10.132.0.34:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-bbcac-predictor-7874fb45b7-bmwwk |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.34:8080: connect: connection refused |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-db836-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-db836-predictor-6996c59df8 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6460c-predictor-5774675f9-x8h72 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6460c-predictor-5d5d94dfb6-btq8d |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6460c-predictor-5d5d94dfb6-btq8d |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-6460c-7c977b8767-wcnvq |
Killing |
Stopping container sequence-graph-6460c | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-db836-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-db836-predictor-598457c8f9 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-db836-predictor-6996c59df8 |
SuccessfulCreate |
Created pod: error-404-isvc-db836-predictor-6996c59df8-gc6q8 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-db836-predictor-598457c8f9-bc6fv |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-db836-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-db836 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-db836": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-db836": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-db836 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-db836": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6460c-predictor-5774675f9-x8h72 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-db836-predictor-598457c8f9 |
SuccessfulCreate |
Created pod: success-200-isvc-db836-predictor-598457c8f9-bc6fv | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-db836 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-db836": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-db836 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-db836": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-db836": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-db836-predictor-6996c59df8-gc6q8 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-db836-predictor-6996c59df8-gc6q8 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1690" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-db836-predictor-598457c8f9-bc6fv |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-db836-predictor-598457c8f9-bc6fv |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-db836-predictor-598457c8f9-bc6fv |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-db836-predictor-598457c8f9-bc6fv |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-db836-predictor-6996c59df8-gc6q8 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-db836-predictor-598457c8f9-bc6fv |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1690" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-db836-predictor-598457c8f9-bc6fv |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
multus |
success-200-isvc-db836-predictor-598457c8f9-bc6fv |
AddedInterface |
Add eth0 [10.132.0.40/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-db836-predictor-6996c59df8-gc6q8 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-db836-predictor-6996c59df8-gc6q8 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-db836-predictor-6996c59df8-gc6q8 |
AddedInterface |
Add eth0 [10.132.0.41/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-db836-predictor-6996c59df8-gc6q8 |
Created |
Created container: kserve-container | |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-bbcac-77d4868ccd-zlxf7 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4517b-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6460c-predictor-5d5d94dfb6-btq8d |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.37:8643/healthz": dial tcp 10.132.0.37:8643: connect: connection refused | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4517b-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-6460c-predictor-5d5d94dfb6-btq8d |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.37:8080: connect: connection refused |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4517b-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4517b-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-4517b-predictor-64b9b9f5bc-9pcrm |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.38:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-4517b-predictor-677799f794-kj8t9 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.39:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-6460c-7c977b8767-wcnvq |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-db836-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-db836-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-db836-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-db836-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-4517b |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-4517b |
InferenceServiceReady |
InferenceService [success-200-isvc-4517b] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-4517b |
InferenceServiceReady |
InferenceService [error-404-isvc-4517b] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-4517b |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-db836-predictor-598457c8f9-bc6fv |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.40:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
splitter-graph-4517b-7b778f989-h6fmq |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "splitter-graph-4517b-serving-cert" not found | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-4517b |
InferenceGraphReady |
InferenceGraph [splitter-graph-4517b] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-4517b-7b778f989 |
SuccessfulCreate |
Created pod: splitter-graph-4517b-7b778f989-h6fmq | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-4517b |
ScalingReplicaSet |
Scaled up replica set splitter-graph-4517b-7b778f989 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
splitter-graph-4517b-7b778f989-h6fmq |
AddedInterface |
Add eth0 [10.134.0.38/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-4517b-7b778f989-h6fmq |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:c0785f60b462f861cfab04ad440fa6adb06dbacec8c1d6b98ebf808ae746d9cf" already present on machine | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-4517b-7b778f989-h6fmq |
Created |
Created container: splitter-graph-4517b | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-4517b-7b778f989-h6fmq |
Started |
Started container splitter-graph-4517b | |
| (x8) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-db836-predictor-6996c59df8-gc6q8 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.41:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-db836 |
InferenceServiceReady |
InferenceService [success-200-isvc-db836] is Ready | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4517b-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4517b-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4517b-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-db836 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4517b-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-bbf0f-predictor-6d7cf8c5c |
SuccessfulCreate |
Created pod: success-200-isvc-bbf0f-predictor-6d7cf8c5c-bc4gn | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-bbf0f-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-bbf0f-predictor-6d7cf8c5c from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4517b-predictor-64b9b9f5bc-9pcrm |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-4517b-7b778f989-h6fmq |
Killing |
Stopping container splitter-graph-4517b | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4517b-predictor-64b9b9f5bc-9pcrm |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-bbf0f |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-bbf0f": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-bbf0f-predictor-85ff6cdb4 |
SuccessfulCreate |
Created pod: error-404-isvc-bbf0f-predictor-85ff6cdb4-jzkmn | |
kserve-ci-e2e-test |
multus |
success-200-isvc-bbf0f-predictor-6d7cf8c5c-bc4gn |
AddedInterface |
Add eth0 [10.132.0.42/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-bbf0f-predictor-6d7cf8c5c-bc4gn |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-bbf0f-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4517b-predictor-677799f794-kj8t9 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-bbf0f-predictor-85ff6cdb4-jzkmn |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-bbf0f-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-bbf0f-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-bbf0f-predictor-85ff6cdb4 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4517b-predictor-677799f794-kj8t9 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-bbf0f |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-bbf0f": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-bbf0f": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-bbf0f-predictor-6d7cf8c5c-bc4gn |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1690" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-bbf0f-predictor-6d7cf8c5c-bc4gn |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4517b-predictor-64b9b9f5bc-9pcrm |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.38:8643/healthz": dial tcp 10.132.0.38:8643: connect: connection refused | |
kserve-ci-e2e-test |
multus |
error-404-isvc-bbf0f-predictor-85ff6cdb4-jzkmn |
AddedInterface |
Add eth0 [10.132.0.43/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-bbf0f-predictor-85ff6cdb4-jzkmn |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1690" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-bbf0f-predictor-85ff6cdb4-jzkmn |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-bbf0f-predictor-85ff6cdb4-jzkmn |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-bbf0f-predictor-85ff6cdb4-jzkmn |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-bbf0f-predictor-85ff6cdb4-jzkmn |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-bbf0f-predictor-6d7cf8c5c-bc4gn |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-bbf0f-predictor-85ff6cdb4-jzkmn |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-bbf0f-predictor-6d7cf8c5c-bc4gn |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-bbf0f-predictor-6d7cf8c5c-bc4gn |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-bbf0f-predictor-6d7cf8c5c-bc4gn |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-db836 |
InferenceServiceReady |
InferenceService [error-404-isvc-db836] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-db836 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4517b-predictor-677799f794-kj8t9 |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.39:8643/healthz": dial tcp 10.132.0.39:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
switch-graph-db836-5758686fdc-568g5 |
Created |
Created container: switch-graph-db836 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-db836-5758686fdc-568g5 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:c0785f60b462f861cfab04ad440fa6adb06dbacec8c1d6b98ebf808ae746d9cf" already present on machine | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-db836-5758686fdc |
SuccessfulCreate |
Created pod: switch-graph-db836-5758686fdc-568g5 | |
kserve-ci-e2e-test |
multus |
switch-graph-db836-5758686fdc-568g5 |
AddedInterface |
Add eth0 [10.134.0.39/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-db836 |
InferenceGraphReady |
InferenceGraph [switch-graph-db836] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-db836 |
ScalingReplicaSet |
Scaled up replica set switch-graph-db836-5758686fdc from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-db836-5758686fdc-568g5 |
Started |
Started container switch-graph-db836 | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-bbf0f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-bbf0f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-db836-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-db836 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-db836 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-db836-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-db836-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-db836-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-4517b-7b778f989-h6fmq |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-db836 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-db836 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-bbf0f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-bbf0f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-bbf0f |
InferenceServiceReady |
InferenceService [error-404-isvc-bbf0f] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-bbf0f |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-bbf0f |
InferenceServiceReady |
InferenceService [success-200-isvc-bbf0f] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-bbf0f |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-bbf0f-c98544745 |
SuccessfulCreate |
Created pod: splitter-graph-bbf0f-c98544745-zdsfs | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-bbf0f |
ScalingReplicaSet |
Scaled up replica set splitter-graph-bbf0f-c98544745 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-bbf0f-c98544745-zdsfs |
Started |
Started container splitter-graph-bbf0f | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-bbf0f-c98544745-zdsfs |
Created |
Created container: splitter-graph-bbf0f | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-bbf0f-c98544745-zdsfs |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:c0785f60b462f861cfab04ad440fa6adb06dbacec8c1d6b98ebf808ae746d9cf" already present on machine | |
kserve-ci-e2e-test |
multus |
splitter-graph-bbf0f-c98544745-zdsfs |
AddedInterface |
Add eth0 [10.134.0.40/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-bbf0f |
InferenceGraphReady |
InferenceGraph [splitter-graph-bbf0f] is Ready | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-bbf0f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-bbf0f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-bbf0f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-bbf0f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-bbf0f |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-bbf0f |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-bbf0f-predictor-6d7cf8c5c-bc4gn |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-bbf0f-predictor-6d7cf8c5c-bc4gn |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-bbf0f-c98544745-zdsfs |
Killing |
Stopping container splitter-graph-bbf0f | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-bbf0f-predictor-85ff6cdb4-jzkmn |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-bbf0f-predictor-85ff6cdb4-jzkmn |
Killing |
Stopping container kube-rbac-proxy | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-bbf0f-predictor-85ff6cdb4-jzkmn |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.43:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-bbf0f-predictor-85ff6cdb4-jzkmn |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.43:8643/healthz": dial tcp 10.132.0.43:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-bbf0f-predictor-6d7cf8c5c-bc4gn |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.42:8643/healthz": dial tcp 10.132.0.42:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-bbf0f-predictor-6d7cf8c5c-bc4gn |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.42:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-bbf0f-c98544745-zdsfs |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
switch-graph-db836-5758686fdc-568g5 |
Killing |
Stopping container switch-graph-db836 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-db836-predictor-6996c59df8-gc6q8 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-db836-predictor-598457c8f9-bc6fv |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-db836-predictor-598457c8f9-bc6fv |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-db836-predictor-6996c59df8-gc6q8 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-db836-predictor-6996c59df8-gc6q8 |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.41:8643/healthz": context deadline exceeded | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-db836-5758686fdc-568g5 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |