| Time | Namespace | Component | RelatedObject | Reason | Message |
|---|---|---|---|---|---|
kserve-ci-e2e-test |
isvc-xgboost-graph-predictor-555fc69d56-bkcxs |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bkcxs to ip-10-0-138-105.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-3f635-predictor-697669ffb4-bjr2d |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-3f635-predictor-697669ffb4-bjr2d to ip-10-0-138-105.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-3af6d-predictor-67876986cd-bjgx2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-3af6d-predictor-67876986cd-bjgx2 to ip-10-0-138-105.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-12d88-predictor-7b774fdf4d-mp9lm |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-12d88-predictor-7b774fdf4d-mp9lm to ip-10-0-138-105.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-ca9bf-58997fc8cf-zg8ss |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss to ip-10-0-128-63.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-a4a3e-76cd97966c-7r5tr |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr to ip-10-0-128-63.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-43d66-predictor-5cccddb6f-zzgq4 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-43d66-predictor-5cccddb6f-zzgq4 to ip-10-0-130-182.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-9f825-7798c68c55-glwtv |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv to ip-10-0-128-63.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-80c39-76464f897d-75xtt |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt to ip-10-0-128-63.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-43d66-7687b75cd7-xvcz2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2 to ip-10-0-128-63.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-9755b-f5958d5c6-nrxg6 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6 to ip-10-0-128-63.ec2.internal | ||
kserve-ci-e2e-test |
model-chainer-fffbcdb68-98rmk |
Scheduled |
Successfully assigned kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk to ip-10-0-128-63.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-a6d98-665c6779cc-k6wxj |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj to ip-10-0-128-63.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-9755b-predictor-58df99b7-jlvhd |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-9755b-predictor-58df99b7-jlvhd to ip-10-0-130-182.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-43d66-predictor-7ff977846d-4qxhj |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-43d66-predictor-7ff977846d-4qxhj to ip-10-0-138-105.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-1-predictor-5579fcbfd8-2dhtw |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5579fcbfd8-2dhtw to ip-10-0-138-105.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-9f825-predictor-847f658579-n7z45 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-9f825-predictor-847f658579-n7z45 to ip-10-0-130-182.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-3f635-predictor-bd67df8df-mxddc |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-3f635-predictor-bd67df8df-mxddc to ip-10-0-130-182.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-80c39-predictor-59cf7d7b96-xp9n5 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-80c39-predictor-59cf7d7b96-xp9n5 to ip-10-0-138-105.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-3af6d-predictor-7dd87fb4f9-zgxk4 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-3af6d-predictor-7dd87fb4f9-zgxk4 to ip-10-0-130-182.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-9755b-predictor-756f474dfd-jjqqk |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-9755b-predictor-756f474dfd-jjqqk to ip-10-0-138-105.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-9f825-predictor-6cfc7876b6-flxxb |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-9f825-predictor-6cfc7876b6-flxxb to ip-10-0-138-105.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-3f635-7c445694c5-k6nq4 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4 to ip-10-0-128-63.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-3af6d-55559fdf56-6zztd |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd to ip-10-0-128-63.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-12d88-predictor-55c899946f-kjljc |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-12d88-predictor-55c899946f-kjljc to ip-10-0-130-182.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-ca9bf-predictor-6bbb4bb6fb-bstjc |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-ca9bf-predictor-6bbb4bb6fb-bstjc to ip-10-0-138-105.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-ca9bf-predictor-5bbff7d989-mn5ll |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-ca9bf-predictor-5bbff7d989-mn5ll to ip-10-0-130-182.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-12d88-7bbf487fc6-js8jk |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk to ip-10-0-128-63.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-a4a3e-predictor-df7cf9c54-h6kkw |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-a4a3e-predictor-df7cf9c54-h6kkw to ip-10-0-138-105.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-2-predictor-f5c56b76d-td6qj |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-f5c56b76d-td6qj to ip-10-0-130-182.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-a6d98-predictor-697b88f8b7-9wscs |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-a6d98-predictor-697b88f8b7-9wscs to ip-10-0-130-182.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-a6d98-predictor-56646fb6cd-8cfd9 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-a6d98-predictor-56646fb6cd-8cfd9 to ip-10-0-138-105.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-80c39-predictor-58b9fd5fdf-kf94j |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-80c39-predictor-58b9fd5fdf-kf94j to ip-10-0-130-182.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-a4a3e-predictor-6667499d67-jztfn |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-a4a3e-predictor-6667499d67-jztfn to ip-10-0-130-182.ec2.internal | ||
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-2-predictor-f5c56b76d |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-2-predictor-f5c56b76d-td6qj | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-a6d98-predictor-56646fb6cd |
SuccessfulCreate |
Created pod: success-200-isvc-a6d98-predictor-56646fb6cd-8cfd9 | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-2-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-2-predictor-f5c56b76d from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
isvc-xgboost-graph-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-xgboost-graph-predictor-555fc69d56 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-xgboost-graph-predictor-555fc69d56 |
SuccessfulCreate |
Created pod: isvc-xgboost-graph-predictor-555fc69d56-bkcxs | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-a6d98-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-a6d98-predictor-697b88f8b7 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-a6d98-predictor-697b88f8b7 |
SuccessfulCreate |
Created pod: error-404-isvc-a6d98-predictor-697b88f8b7-9wscs | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-1-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-1-predictor-5579fcbfd8 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-1-predictor-5579fcbfd8 |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-1-predictor-5579fcbfd8-2dhtw | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-a6d98 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-a6d98": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-a6d98": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-a6d98-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-a6d98-predictor-56646fb6cd from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-a6d98 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-a6d98": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
multus |
success-200-isvc-a6d98-predictor-56646fb6cd-8cfd9 |
AddedInterface |
Add eth0 [10.132.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-bkcxs |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-5579fcbfd8-2dhtw |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-f5c56b76d-td6qj |
Pulling |
Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:10dce2bd6581ed3398139cc0e491de5addc169f1b70e0e912dc9be169495c7fb" | |
kserve-ci-e2e-test |
multus |
error-404-isvc-a6d98-predictor-697b88f8b7-9wscs |
AddedInterface |
Add eth0 [10.133.0.20/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
isvc-xgboost-graph-predictor-555fc69d56-bkcxs |
AddedInterface |
Add eth0 [10.132.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-bkcxs |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:10dce2bd6581ed3398139cc0e491de5addc169f1b70e0e912dc9be169495c7fb" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-bkcxs |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-5579fcbfd8-2dhtw |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-5579fcbfd8-2dhtw |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:10dce2bd6581ed3398139cc0e491de5addc169f1b70e0e912dc9be169495c7fb" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a6d98-predictor-697b88f8b7-9wscs |
Pulling |
Pulling image "quay.io/opendatahub/error-404-isvc:odh-pr-1947" | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-2-predictor-f5c56b76d-td6qj |
AddedInterface |
Add eth0 [10.133.0.21/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-1-predictor-5579fcbfd8-2dhtw |
AddedInterface |
Add eth0 [10.132.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a6d98-predictor-56646fb6cd-8cfd9 |
Pulling |
Pulling image "quay.io/opendatahub/success-200-isvc:odh-pr-1947" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-5579fcbfd8-2dhtw |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1947" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-bkcxs |
Pulling |
Pulling image "kserve/xgbserver:latest" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-f5c56b76d-td6qj |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:10dce2bd6581ed3398139cc0e491de5addc169f1b70e0e912dc9be169495c7fb" in 11.059s (11.059s including waiting). Image size: 308281769 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a6d98-predictor-56646fb6cd-8cfd9 |
Pulled |
Successfully pulled image "quay.io/opendatahub/success-200-isvc:odh-pr-1947" in 12.092s (12.092s including waiting). Image size: 1356691533 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a6d98-predictor-56646fb6cd-8cfd9 |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a6d98-predictor-56646fb6cd-8cfd9 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a6d98-predictor-56646fb6cd-8cfd9 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a6d98-predictor-697b88f8b7-9wscs |
Pulled |
Successfully pulled image "quay.io/opendatahub/error-404-isvc:odh-pr-1947" in 12.981s (12.981s including waiting). Image size: 1355773512 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-f5c56b76d-td6qj |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-f5c56b76d-td6qj |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a6d98-predictor-697b88f8b7-9wscs |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a6d98-predictor-697b88f8b7-9wscs |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a6d98-predictor-697b88f8b7-9wscs |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a6d98-predictor-697b88f8b7-9wscs |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a6d98-predictor-697b88f8b7-9wscs |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a6d98-predictor-697b88f8b7-9wscs |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.241s (2.241s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-f5c56b76d-td6qj |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1947" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-5579fcbfd8-2dhtw |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a6d98-predictor-56646fb6cd-8cfd9 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-5579fcbfd8-2dhtw |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-5579fcbfd8-2dhtw |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1947" in 14.18s (14.18s including waiting). Image size: 1580720527 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a6d98-predictor-56646fb6cd-8cfd9 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a6d98-predictor-56646fb6cd-8cfd9 |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 7.289s (7.289s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-5579fcbfd8-2dhtw |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-5579fcbfd8-2dhtw |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-5579fcbfd8-2dhtw |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-f5c56b76d-td6qj |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-f5c56b76d-td6qj |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-f5c56b76d-td6qj |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1947" in 5.931s (5.931s including waiting). Image size: 1580720527 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-f5c56b76d-td6qj |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-f5c56b76d-td6qj |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-f5c56b76d-td6qj |
Started |
Started container kube-rbac-proxy | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-bkcxs |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-bkcxs |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-bkcxs |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-bkcxs |
Pulled |
Successfully pulled image "kserve/xgbserver:latest" in 25.545s (25.545s including waiting). Image size: 1332293378 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-bkcxs |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-bkcxs |
Created |
Created container: kube-rbac-proxy | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-a6d98-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-a6d98-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-a6d98-predictor-56646fb6cd-8cfd9 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.33:8080: connect: connection refused |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-a6d98-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-a6d98-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-a6d98 |
InferenceServiceReady |
InferenceService [error-404-isvc-a6d98] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-a6d98 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-a6d98 |
InferenceServiceReady |
InferenceService [success-200-isvc-a6d98] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-a6d98 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-a6d98-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-a6d98-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-a6d98-665c6779cc |
SuccessfulCreate |
Created pod: switch-graph-a6d98-665c6779cc-k6wxj | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-a6d98 |
ScalingReplicaSet |
Scaled up replica set switch-graph-a6d98-665c6779cc from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-a6d98 |
InferenceGraphReady |
InferenceGraph [switch-graph-a6d98] is Ready | |
kserve-ci-e2e-test |
multus |
switch-graph-a6d98-665c6779cc-k6wxj |
AddedInterface |
Add eth0 [10.134.0.22/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-a6d98-665c6779cc-k6wxj |
Pulling |
Pulling image "quay.io/opendatahub/kserve-router@sha256:ca6150683b88d0bc774f8e42803fb5db7032d026ca2a7ea35068f270ce0317f1" | |
kserve-ci-e2e-test |
kubelet |
switch-graph-a6d98-665c6779cc-k6wxj |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-router@sha256:ca6150683b88d0bc774f8e42803fb5db7032d026ca2a7ea35068f270ce0317f1" in 2.822s (2.822s including waiting). Image size: 214921807 bytes. | |
kserve-ci-e2e-test |
kubelet |
switch-graph-a6d98-665c6779cc-k6wxj |
Created |
Created container: switch-graph-a6d98 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-a6d98-665c6779cc-k6wxj |
Started |
Started container switch-graph-a6d98 | |
| (x8) | kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-bkcxs |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.35:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a6d98-predictor-56646fb6cd-8cfd9 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-3f635-predictor-697669ffb4 |
SuccessfulCreate |
Created pod: success-200-isvc-3f635-predictor-697669ffb4-bjr2d | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-a6d98-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-a6d98-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-3f635-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-3f635-predictor-697669ffb4 from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-3f635-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-3f635-predictor-bd67df8df from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a6d98-predictor-56646fb6cd-8cfd9 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a6d98-predictor-697b88f8b7-9wscs |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a6d98-predictor-697b88f8b7-9wscs |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
switch-graph-a6d98-665c6779cc-k6wxj |
Killing |
Stopping container switch-graph-a6d98 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3f635-predictor-697669ffb4-bjr2d |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-3f635-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3f635-predictor-bd67df8df-mxddc |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-3f635-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-3f635-predictor-bd67df8df |
SuccessfulCreate |
Created pod: error-404-isvc-3f635-predictor-bd67df8df-mxddc | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3f635-predictor-bd67df8df-mxddc |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3f635-predictor-bd67df8df-mxddc |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3f635-predictor-697669ffb4-bjr2d |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1947" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3f635-predictor-697669ffb4-bjr2d |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3f635-predictor-bd67df8df-mxddc |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a6d98-predictor-56646fb6cd-8cfd9 |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.33:8643/healthz": dial tcp 10.132.0.33:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3f635-predictor-bd67df8df-mxddc |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-3f635-predictor-697669ffb4-bjr2d |
AddedInterface |
Add eth0 [10.132.0.36/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3f635-predictor-bd67df8df-mxddc |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3f635-predictor-bd67df8df-mxddc |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1947" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-3f635-predictor-bd67df8df-mxddc |
AddedInterface |
Add eth0 [10.133.0.22/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3f635-predictor-697669ffb4-bjr2d |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3f635-predictor-697669ffb4-bjr2d |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3f635-predictor-697669ffb4-bjr2d |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3f635-predictor-697669ffb4-bjr2d |
Created |
Created container: kube-rbac-proxy | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-a6d98-predictor-697b88f8b7-9wscs |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.20:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a6d98-predictor-697b88f8b7-9wscs |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.20:8643/healthz": dial tcp 10.133.0.20:8643: connect: connection refused | |
| (x10) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-5579fcbfd8-2dhtw |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.34:8080: connect: connection refused |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
InferenceServiceReady |
InferenceService [isvc-xgboost-graph] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-2] is Ready | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-1] is Ready | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-a6d98-665c6779cc-k6wxj |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-3f635-predictor-697669ffb4-bjr2d |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.36:8080: connect: connection refused |
kserve-ci-e2e-test |
InferenceGraphController |
model-chainer |
InferenceGraphReady |
InferenceGraph [model-chainer] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
model-chainer |
ScalingReplicaSet |
Scaled up replica set model-chainer-fffbcdb68 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
model-chainer-fffbcdb68 |
SuccessfulCreate |
Created pod: model-chainer-fffbcdb68-98rmk | |
kserve-ci-e2e-test |
kubelet |
model-chainer-fffbcdb68-98rmk |
Started |
Started container model-chainer | |
kserve-ci-e2e-test |
kubelet |
model-chainer-fffbcdb68-98rmk |
Created |
Created container: model-chainer | |
kserve-ci-e2e-test |
multus |
model-chainer-fffbcdb68-98rmk |
AddedInterface |
Add eth0 [10.134.0.23/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
model-chainer-fffbcdb68-98rmk |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:ca6150683b88d0bc774f8e42803fb5db7032d026ca2a7ea35068f270ce0317f1" already present on machine | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-3f635-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x7) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x7) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-3f635-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-3f635-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-3f635-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-3f635 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-3f635 |
InferenceServiceReady |
InferenceService [error-404-isvc-3f635] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-3f635 |
InferenceServiceReady |
InferenceService [success-200-isvc-3f635] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-3f635 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
kubelet |
model-chainer-fffbcdb68-98rmk |
Killing |
Stopping container model-chainer | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-bkcxs |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-f5c56b76d-td6qj |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-f5c56b76d-td6qj |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-9f825-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-9f825-predictor-6cfc7876b6 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-5579fcbfd8-2dhtw |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-5579fcbfd8-2dhtw |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-9f825-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-9f825-predictor-847f658579 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-9f825-predictor-847f658579 |
SuccessfulCreate |
Created pod: error-404-isvc-9f825-predictor-847f658579-n7z45 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-9f825-predictor-6cfc7876b6 |
SuccessfulCreate |
Created pod: success-200-isvc-9f825-predictor-6cfc7876b6-flxxb | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-bkcxs |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9f825-predictor-847f658579-n7z45 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9f825-predictor-6cfc7876b6-flxxb |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9f825-predictor-847f658579-n7z45 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9f825-predictor-847f658579-n7z45 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1947" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-9f825-predictor-847f658579-n7z45 |
AddedInterface |
Add eth0 [10.133.0.23/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9f825-predictor-847f658579-n7z45 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9f825-predictor-847f658579-n7z45 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9f825-predictor-6cfc7876b6-flxxb |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9f825-predictor-6cfc7876b6-flxxb |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9f825-predictor-6cfc7876b6-flxxb |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9f825-predictor-847f658579-n7z45 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9f825-predictor-6cfc7876b6-flxxb |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9f825-predictor-6cfc7876b6-flxxb |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1947" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-9f825-predictor-6cfc7876b6-flxxb |
AddedInterface |
Add eth0 [10.132.0.37/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-5579fcbfd8-2dhtw |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.34:8643/healthz": dial tcp 10.132.0.34:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-bkcxs |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.35:8643/healthz": dial tcp 10.132.0.35:8643: connect: connection refused | |
| (x11) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-f5c56b76d-td6qj |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.21:8080: connect: connection refused |
| (x2) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-f5c56b76d-td6qj |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.21:8643/healthz": dial tcp 10.133.0.21:8643: connect: connection refused |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-3f635-7c445694c5 |
SuccessfulCreate |
Created pod: switch-graph-3f635-7c445694c5-k6nq4 | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-3f635 |
ScalingReplicaSet |
Scaled up replica set switch-graph-3f635-7c445694c5 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-3f635 |
InferenceGraphReady |
InferenceGraph [switch-graph-3f635] is Ready | |
kserve-ci-e2e-test |
multus |
switch-graph-3f635-7c445694c5-k6nq4 |
AddedInterface |
Add eth0 [10.134.0.24/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-3f635-7c445694c5-k6nq4 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:ca6150683b88d0bc774f8e42803fb5db7032d026ca2a7ea35068f270ce0317f1" already present on machine | |
kserve-ci-e2e-test |
kubelet |
switch-graph-3f635-7c445694c5-k6nq4 |
Started |
Started container switch-graph-3f635 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-3f635-7c445694c5-k6nq4 |
Created |
Created container: switch-graph-3f635 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-3f635-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-3f635-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-3f635 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-3f635 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-3f635-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-3f635-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
model-chainer-fffbcdb68-98rmk |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-9f825-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-9f825-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-9f825-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-9f825-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-9f825-predictor-6cfc7876b6-flxxb |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.37:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-9f825-predictor-847f658579-n7z45 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.23:8080: connect: connection refused |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-3f635 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-3f635 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-9f825 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-9f825 |
InferenceServiceReady |
InferenceService [success-200-isvc-9f825] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-9f825 |
InferenceServiceReady |
InferenceService [error-404-isvc-9f825] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-9f825 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-9f825-7798c68c55 |
SuccessfulCreate |
Created pod: sequence-graph-9f825-7798c68c55-glwtv | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-9f825 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-9f825-7798c68c55 from 0 to 1 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-9f825-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-9f825 |
InferenceGraphReady |
InferenceGraph [sequence-graph-9f825] is Ready | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-9f825 |
UpdateFailed |
Failed to update status for InferenceGraph "sequence-graph-9f825": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "sequence-graph-9f825": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-9f825-7798c68c55-glwtv |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "sequence-graph-9f825-serving-cert" not found | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-9f825 |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "sequence-graph-9f825": the object has been modified; please apply your changes to the latest version and try again | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-9f825-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
sequence-graph-9f825-7798c68c55-glwtv |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:ca6150683b88d0bc774f8e42803fb5db7032d026ca2a7ea35068f270ce0317f1" already present on machine | |
kserve-ci-e2e-test |
multus |
sequence-graph-9f825-7798c68c55-glwtv |
AddedInterface |
Add eth0 [10.134.0.25/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-9f825-7798c68c55-glwtv |
Created |
Created container: sequence-graph-9f825 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-9f825-7798c68c55-glwtv |
Started |
Started container sequence-graph-9f825 | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-9f825-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-9f825-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-9f825 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-9f825 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-12d88 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-12d88": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-12d88-predictor-7b774fdf4d |
SuccessfulCreate |
Created pod: success-200-isvc-12d88-predictor-7b774fdf4d-mp9lm | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-12d88 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-12d88": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-12d88": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-12d88-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-12d88-predictor-55c899946f from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-12d88-predictor-55c899946f |
SuccessfulCreate |
Created pod: error-404-isvc-12d88-predictor-55c899946f-kjljc | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3f635-predictor-697669ffb4-bjr2d |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3f635-predictor-697669ffb4-bjr2d |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3f635-predictor-bd67df8df-mxddc |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3f635-predictor-bd67df8df-mxddc |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-12d88-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-12d88-predictor-7b774fdf4d from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-12d88 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-12d88": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-12d88 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-12d88": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-12d88": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
switch-graph-3f635-7c445694c5-k6nq4 |
Killing |
Stopping container switch-graph-3f635 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-12d88-predictor-55c899946f-kjljc |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-12d88-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-12d88-predictor-7b774fdf4d-mp9lm |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-12d88-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
multus |
success-200-isvc-12d88-predictor-7b774fdf4d-mp9lm |
AddedInterface |
Add eth0 [10.132.0.38/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-12d88-predictor-7b774fdf4d-mp9lm |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1947" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-12d88-predictor-55c899946f-kjljc |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-12d88-predictor-7b774fdf4d-mp9lm |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-12d88-predictor-55c899946f-kjljc |
AddedInterface |
Add eth0 [10.133.0.24/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-12d88-predictor-55c899946f-kjljc |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-12d88-predictor-55c899946f-kjljc |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-12d88-predictor-55c899946f-kjljc |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-12d88-predictor-55c899946f-kjljc |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-12d88-predictor-7b774fdf4d-mp9lm |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-12d88-predictor-55c899946f-kjljc |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1947" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-12d88-predictor-7b774fdf4d-mp9lm |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-12d88-predictor-7b774fdf4d-mp9lm |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-12d88-predictor-7b774fdf4d-mp9lm |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-3f635-predictor-bd67df8df-mxddc |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.22:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3f635-predictor-bd67df8df-mxddc |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.22:8643/healthz": dial tcp 10.133.0.22:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-3f635-7c445694c5-k6nq4 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-12d88-predictor-55c899946f-kjljc |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.24:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-12d88-predictor-7b774fdf4d-mp9lm |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.38:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-12d88-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-12d88-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-12d88-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-12d88-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-12d88 |
InferenceServiceReady |
InferenceService [success-200-isvc-12d88] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-12d88 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-12d88 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-12d88 |
InferenceServiceReady |
InferenceService [error-404-isvc-12d88] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-80c39-predictor-58b9fd5fdf |
SuccessfulCreate |
Created pod: error-404-isvc-80c39-predictor-58b9fd5fdf-kf94j | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-80c39-predictor-59cf7d7b96 |
SuccessfulCreate |
Created pod: success-200-isvc-80c39-predictor-59cf7d7b96-xp9n5 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-9f825-7798c68c55-glwtv |
Killing |
Stopping container sequence-graph-9f825 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9f825-predictor-847f658579-n7z45 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9f825-predictor-847f658579-n7z45 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9f825-predictor-6cfc7876b6-flxxb |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-80c39-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-80c39-predictor-58b9fd5fdf from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9f825-predictor-6cfc7876b6-flxxb |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-80c39-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-80c39-predictor-59cf7d7b96 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-80c39-predictor-58b9fd5fdf-kf94j |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-80c39-predictor-59cf7d7b96-xp9n5 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-80c39-predictor-58b9fd5fdf-kf94j |
AddedInterface |
Add eth0 [10.133.0.25/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-80c39-predictor-58b9fd5fdf-kf94j |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-80c39-predictor-58b9fd5fdf-kf94j |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-80c39-predictor-58b9fd5fdf-kf94j |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-80c39-predictor-58b9fd5fdf-kf94j |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-80c39-predictor-59cf7d7b96-xp9n5 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-80c39-predictor-59cf7d7b96-xp9n5 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-80c39-predictor-59cf7d7b96-xp9n5 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
multus |
success-200-isvc-80c39-predictor-59cf7d7b96-xp9n5 |
AddedInterface |
Add eth0 [10.132.0.39/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-80c39-predictor-59cf7d7b96-xp9n5 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1947" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-80c39-predictor-59cf7d7b96-xp9n5 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-80c39-predictor-58b9fd5fdf-kf94j |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1947" already present on machine | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-12d88-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-12d88 |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-12d88-7bbf487fc6 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-12d88 |
InferenceGraphReady |
InferenceGraph [ensemble-graph-12d88] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-12d88-7bbf487fc6 |
SuccessfulCreate |
Created pod: ensemble-graph-12d88-7bbf487fc6-js8jk | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-12d88-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-12d88-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-12d88-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-12d88-7bbf487fc6-js8jk |
Started |
Started container ensemble-graph-12d88 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-12d88-7bbf487fc6-js8jk |
Created |
Created container: ensemble-graph-12d88 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-12d88-7bbf487fc6-js8jk |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:ca6150683b88d0bc774f8e42803fb5db7032d026ca2a7ea35068f270ce0317f1" already present on machine | |
kserve-ci-e2e-test |
multus |
ensemble-graph-12d88-7bbf487fc6-js8jk |
AddedInterface |
Add eth0 [10.134.0.26/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-80c39-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-80c39-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-12d88-predictor-7b774fdf4d-mp9lm |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-12d88-predictor-7b774fdf4d-mp9lm |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-12d88-7bbf487fc6-js8jk |
Killing |
Stopping container ensemble-graph-12d88 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-12d88-predictor-55c899946f-kjljc |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-12d88-predictor-55c899946f-kjljc |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9755b-predictor-58df99b7-jlvhd |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9755b-predictor-58df99b7-jlvhd |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-9755b-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-9755b-predictor-756f474dfd from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-9755b-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-9755b-predictor-58df99b7 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-9755b-predictor-756f474dfd |
SuccessfulCreate |
Created pod: success-200-isvc-9755b-predictor-756f474dfd-jjqqk | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-9755b-predictor-58df99b7 |
SuccessfulCreate |
Created pod: error-404-isvc-9755b-predictor-58df99b7-jlvhd | |
kserve-ci-e2e-test |
multus |
error-404-isvc-9755b-predictor-58df99b7-jlvhd |
AddedInterface |
Add eth0 [10.133.0.26/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9755b-predictor-58df99b7-jlvhd |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9755b-predictor-58df99b7-jlvhd |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1947" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9755b-predictor-756f474dfd-jjqqk |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-9755b-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9755b-predictor-58df99b7-jlvhd |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9755b-predictor-58df99b7-jlvhd |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9755b-predictor-756f474dfd-jjqqk |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9755b-predictor-756f474dfd-jjqqk |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9755b-predictor-756f474dfd-jjqqk |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9755b-predictor-756f474dfd-jjqqk |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9755b-predictor-756f474dfd-jjqqk |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9755b-predictor-756f474dfd-jjqqk |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1947" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-9755b-predictor-756f474dfd-jjqqk |
AddedInterface |
Add eth0 [10.132.0.40/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-12d88-predictor-55c899946f-kjljc |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.24:8643/healthz": dial tcp 10.133.0.24:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-9f825-7798c68c55-glwtv |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-80c39-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-80c39-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-80c39-predictor-59cf7d7b96-xp9n5 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.39:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-80c39-predictor-58b9fd5fdf-kf94j |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.25:8080: connect: connection refused |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-80c39 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-80c39 |
InferenceServiceReady |
InferenceService [success-200-isvc-80c39] is Ready | |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-12d88-7bbf487fc6-js8jk |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-80c39 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-80c39 |
InferenceServiceReady |
InferenceService [error-404-isvc-80c39] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-9755b-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-9755b-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-9755b-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-9755b-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-80c39-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-80c39-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-80c39-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-80c39-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-80c39-76464f897d |
SuccessfulCreate |
Created pod: sequence-graph-80c39-76464f897d-75xtt | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-80c39 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-80c39-76464f897d from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-80c39-76464f897d-75xtt |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "sequence-graph-80c39-serving-cert" not found | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-80c39 |
InferenceGraphReady |
InferenceGraph [sequence-graph-80c39] is Ready | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-80c39-76464f897d-75xtt |
Started |
Started container sequence-graph-80c39 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-80c39-76464f897d-75xtt |
Created |
Created container: sequence-graph-80c39 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-80c39-76464f897d-75xtt |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:ca6150683b88d0bc774f8e42803fb5db7032d026ca2a7ea35068f270ce0317f1" already present on machine | |
kserve-ci-e2e-test |
multus |
sequence-graph-80c39-76464f897d-75xtt |
AddedInterface |
Add eth0 [10.134.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-9755b |
InferenceServiceReady |
InferenceService [success-200-isvc-9755b] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-9755b |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-9755b |
InferenceServiceReady |
InferenceService [error-404-isvc-9755b] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-9755b |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-43d66-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-43d66-predictor-7ff977846d from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-80c39-predictor-58b9fd5fdf-kf94j |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-80c39-predictor-58b9fd5fdf-kf94j |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-43d66-predictor-7ff977846d |
SuccessfulCreate |
Created pod: success-200-isvc-43d66-predictor-7ff977846d-4qxhj | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-80c39-predictor-59cf7d7b96-xp9n5 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-80c39-predictor-59cf7d7b96-xp9n5 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-80c39-76464f897d-75xtt |
Killing |
Stopping container sequence-graph-80c39 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-43d66-predictor-5cccddb6f-zzgq4 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-43d66-predictor-5cccddb6f-zzgq4 |
AddedInterface |
Add eth0 [10.133.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-43d66-predictor-5cccddb6f |
SuccessfulCreate |
Created pod: error-404-isvc-43d66-predictor-5cccddb6f-zzgq4 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-43d66-predictor-5cccddb6f-zzgq4 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-43d66-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-43d66-predictor-5cccddb6f from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-43d66-predictor-5cccddb6f-zzgq4 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-43d66-predictor-5cccddb6f-zzgq4 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-43d66-predictor-5cccddb6f-zzgq4 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-43d66-predictor-5cccddb6f-zzgq4 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1947" already present on machine | |
| (x2) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-43d66-predictor-7ff977846d-4qxhj |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-43d66-predictor-serving-cert" not found |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-80c39-predictor-59cf7d7b96-xp9n5 |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.39:8643/healthz": dial tcp 10.132.0.39:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-43d66-predictor-7ff977846d-4qxhj |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-43d66-predictor-7ff977846d-4qxhj |
AddedInterface |
Add eth0 [10.132.0.41/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-43d66-predictor-7ff977846d-4qxhj |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1947" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-43d66-predictor-7ff977846d-4qxhj |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-43d66-predictor-7ff977846d-4qxhj |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-43d66-predictor-7ff977846d-4qxhj |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-43d66-predictor-7ff977846d-4qxhj |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-9755b-f5958d5c6 |
SuccessfulCreate |
Created pod: ensemble-graph-9755b-f5958d5c6-nrxg6 | |
kserve-ci-e2e-test |
multus |
ensemble-graph-9755b-f5958d5c6-nrxg6 |
AddedInterface |
Add eth0 [10.134.0.28/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-9755b |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-9755b-f5958d5c6 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-9755b |
InferenceGraphReady |
InferenceGraph [ensemble-graph-9755b] is Ready | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-9755b-f5958d5c6-nrxg6 |
Started |
Started container ensemble-graph-9755b | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-9755b-f5958d5c6-nrxg6 |
Created |
Created container: ensemble-graph-9755b | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-9755b-f5958d5c6-nrxg6 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:ca6150683b88d0bc774f8e42803fb5db7032d026ca2a7ea35068f270ce0317f1" already present on machine | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-43d66-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-43d66-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-9755b-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-9755b-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-9755b |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-9755b |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-80c39-76464f897d-75xtt |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-43d66-predictor-7ff977846d-4qxhj |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.41:8080: connect: connection refused |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-9755b-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-9755b-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-9755b |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-9755b |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-43d66-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-43d66-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-43d66 |
InferenceServiceReady |
InferenceService [error-404-isvc-43d66] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-43d66 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-43d66 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-43d66 |
InferenceServiceReady |
InferenceService [success-200-isvc-43d66] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-43d66 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-43d66-7687b75cd7 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-43d66-7687b75cd7-xvcz2 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:ca6150683b88d0bc774f8e42803fb5db7032d026ca2a7ea35068f270ce0317f1" already present on machine | |
kserve-ci-e2e-test |
multus |
sequence-graph-43d66-7687b75cd7-xvcz2 |
AddedInterface |
Add eth0 [10.134.0.29/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-43d66-7687b75cd7-xvcz2 |
Created |
Created container: sequence-graph-43d66 | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-43d66 |
InferenceGraphReady |
InferenceGraph [sequence-graph-43d66] is Ready | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-43d66-7687b75cd7-xvcz2 |
Started |
Started container sequence-graph-43d66 | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-43d66-7687b75cd7 |
SuccessfulCreate |
Created pod: sequence-graph-43d66-7687b75cd7-xvcz2 | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-43d66-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-43d66-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-43d66 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-43d66 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-43d66 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-43d66 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-43d66-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-43d66-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-ca9bf-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-ca9bf-predictor-6bbb4bb6fb from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-9755b-f5958d5c6-nrxg6 |
Killing |
Stopping container ensemble-graph-9755b | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-ca9bf-predictor-6bbb4bb6fb |
SuccessfulCreate |
Created pod: success-200-isvc-ca9bf-predictor-6bbb4bb6fb-bstjc | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ca9bf-predictor-6bbb4bb6fb-bstjc |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-ca9bf-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9755b-predictor-58df99b7-jlvhd |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9755b-predictor-756f474dfd-jjqqk |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9755b-predictor-756f474dfd-jjqqk |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-ca9bf-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-ca9bf-predictor-5bbff7d989 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-ca9bf-predictor-5bbff7d989 |
SuccessfulCreate |
Created pod: error-404-isvc-ca9bf-predictor-5bbff7d989-mn5ll | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9755b-predictor-58df99b7-jlvhd |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ca9bf-predictor-5bbff7d989-mn5ll |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-ca9bf-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ca9bf-predictor-5bbff7d989-mn5ll |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ca9bf-predictor-5bbff7d989-mn5ll |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ca9bf-predictor-6bbb4bb6fb-bstjc |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-ca9bf-predictor-6bbb4bb6fb-bstjc |
AddedInterface |
Add eth0 [10.132.0.42/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ca9bf-predictor-6bbb4bb6fb-bstjc |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1947" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ca9bf-predictor-6bbb4bb6fb-bstjc |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ca9bf-predictor-5bbff7d989-mn5ll |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ca9bf-predictor-5bbff7d989-mn5ll |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ca9bf-predictor-6bbb4bb6fb-bstjc |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ca9bf-predictor-5bbff7d989-mn5ll |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ca9bf-predictor-6bbb4bb6fb-bstjc |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ca9bf-predictor-5bbff7d989-mn5ll |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1947" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ca9bf-predictor-6bbb4bb6fb-bstjc |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-ca9bf-predictor-5bbff7d989-mn5ll |
AddedInterface |
Add eth0 [10.133.0.28/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9755b-predictor-756f474dfd-jjqqk |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.40:8643/healthz": dial tcp 10.132.0.40:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-9755b-predictor-756f474dfd-jjqqk |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.40:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-9755b-predictor-58df99b7-jlvhd |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.26:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9755b-predictor-58df99b7-jlvhd |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.26:8643/healthz": dial tcp 10.133.0.26:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-9755b-f5958d5c6-nrxg6 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-ca9bf-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-ca9bf-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-ca9bf-predictor-6bbb4bb6fb-bstjc |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.42:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-ca9bf-predictor-5bbff7d989-mn5ll |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.28:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-ca9bf-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-ca9bf-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-ca9bf |
InferenceServiceReady |
InferenceService [success-200-isvc-ca9bf] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-ca9bf |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-ca9bf |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-ca9bf |
InferenceServiceReady |
InferenceService [error-404-isvc-ca9bf] is Ready | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-43d66-predictor-5cccddb6f-zzgq4 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-3af6d-predictor-67876986cd |
SuccessfulCreate |
Created pod: success-200-isvc-3af6d-predictor-67876986cd-bjgx2 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-43d66-predictor-7ff977846d-4qxhj |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-43d66-predictor-5cccddb6f-zzgq4 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-43d66-predictor-7ff977846d-4qxhj |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-43d66-7687b75cd7-xvcz2 |
Killing |
Stopping container sequence-graph-43d66 | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-3af6d-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-3af6d-predictor-67876986cd from 0 to 1 | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-ca9bf-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-3af6d-predictor-7dd87fb4f9 |
SuccessfulCreate |
Created pod: error-404-isvc-3af6d-predictor-7dd87fb4f9-zgxk4 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3af6d-predictor-7dd87fb4f9-zgxk4 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-3af6d-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3af6d-predictor-67876986cd-bjgx2 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-3af6d-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-ca9bf |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "splitter-graph-ca9bf": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-ca9bf |
UpdateFailed |
Failed to update status for InferenceGraph "splitter-graph-ca9bf": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "splitter-graph-ca9bf": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-3af6d-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-3af6d-predictor-7dd87fb4f9 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-ca9bf-58997fc8cf-zg8ss |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "splitter-graph-ca9bf-serving-cert" not found | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-ca9bf |
InferenceGraphReady |
InferenceGraph [splitter-graph-ca9bf] is Ready | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-43d66-predictor-7ff977846d-4qxhj |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.41:8643/healthz": dial tcp 10.132.0.41:8643: connect: connection refused | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-ca9bf-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-ca9bf-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-ca9bf |
ScalingReplicaSet |
Scaled up replica set splitter-graph-ca9bf-58997fc8cf from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-ca9bf-58997fc8cf |
SuccessfulCreate |
Created pod: splitter-graph-ca9bf-58997fc8cf-zg8ss | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-ca9bf-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-ca9bf-58997fc8cf-zg8ss |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:ca6150683b88d0bc774f8e42803fb5db7032d026ca2a7ea35068f270ce0317f1" already present on machine | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-ca9bf-58997fc8cf-zg8ss |
Created |
Created container: splitter-graph-ca9bf | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3af6d-predictor-67876986cd-bjgx2 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3af6d-predictor-7dd87fb4f9-zgxk4 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
splitter-graph-ca9bf-58997fc8cf-zg8ss |
AddedInterface |
Add eth0 [10.134.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3af6d-predictor-7dd87fb4f9-zgxk4 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3af6d-predictor-7dd87fb4f9-zgxk4 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3af6d-predictor-67876986cd-bjgx2 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1947" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3af6d-predictor-7dd87fb4f9-zgxk4 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
success-200-isvc-3af6d-predictor-67876986cd-bjgx2 |
AddedInterface |
Add eth0 [10.132.0.43/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-ca9bf-58997fc8cf-zg8ss |
Started |
Started container splitter-graph-ca9bf | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3af6d-predictor-67876986cd-bjgx2 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3af6d-predictor-7dd87fb4f9-zgxk4 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-3af6d-predictor-7dd87fb4f9-zgxk4 |
AddedInterface |
Add eth0 [10.133.0.29/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3af6d-predictor-7dd87fb4f9-zgxk4 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1947" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3af6d-predictor-67876986cd-bjgx2 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3af6d-predictor-67876986cd-bjgx2 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3af6d-predictor-67876986cd-bjgx2 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-43d66-predictor-5cccddb6f-zzgq4 |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.27:8643/healthz": dial tcp 10.133.0.27:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-43d66-predictor-5cccddb6f-zzgq4 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.27:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a4a3e-predictor-df7cf9c54-h6kkw |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1947" already present on machine | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-a4a3e-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-a4a3e-predictor-df7cf9c54 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-a4a3e-predictor-df7cf9c54 |
SuccessfulCreate |
Created pod: success-200-isvc-a4a3e-predictor-df7cf9c54-h6kkw | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ca9bf-predictor-5bbff7d989-mn5ll |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ca9bf-predictor-5bbff7d989-mn5ll |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-a4a3e-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-a4a3e-predictor-6667499d67 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-a4a3e-predictor-6667499d67 |
SuccessfulCreate |
Created pod: error-404-isvc-a4a3e-predictor-6667499d67-jztfn | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ca9bf-predictor-6bbb4bb6fb-bstjc |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-ca9bf-58997fc8cf-zg8ss |
Killing |
Stopping container splitter-graph-ca9bf | |
kserve-ci-e2e-test |
multus |
success-200-isvc-a4a3e-predictor-df7cf9c54-h6kkw |
AddedInterface |
Add eth0 [10.132.0.44/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ca9bf-predictor-6bbb4bb6fb-bstjc |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a4a3e-predictor-df7cf9c54-h6kkw |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a4a3e-predictor-df7cf9c54-h6kkw |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a4a3e-predictor-df7cf9c54-h6kkw |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a4a3e-predictor-df7cf9c54-h6kkw |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a4a3e-predictor-df7cf9c54-h6kkw |
Started |
Started container kube-rbac-proxy | |
| (x2) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-a4a3e-predictor-6667499d67-jztfn |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-a4a3e-predictor-serving-cert" not found |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a4a3e-predictor-6667499d67-jztfn |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1947" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a4a3e-predictor-6667499d67-jztfn |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a4a3e-predictor-6667499d67-jztfn |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a4a3e-predictor-6667499d67-jztfn |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-a4a3e-predictor-6667499d67-jztfn |
AddedInterface |
Add eth0 [10.133.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a4a3e-predictor-6667499d67-jztfn |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a4a3e-predictor-6667499d67-jztfn |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ca9bf-predictor-6bbb4bb6fb-bstjc |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.42:8643/healthz": dial tcp 10.132.0.42:8643: connect: connection refused | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-3af6d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-3af6d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-43d66-7687b75cd7-xvcz2 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-3af6d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-3af6d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-ca9bf-58997fc8cf-zg8ss |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-3af6d-predictor-67876986cd-bjgx2 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.43:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-3af6d-predictor-7dd87fb4f9-zgxk4 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.29:8080: connect: connection refused |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-a4a3e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-a4a3e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-a4a3e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-a4a3e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-3af6d |
InferenceServiceReady |
InferenceService [success-200-isvc-3af6d] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-3af6d |
InferenceServiceReady |
InferenceService [error-404-isvc-3af6d] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-3af6d |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-3af6d |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-a4a3e-predictor-6667499d67-jztfn |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.30:8080: connect: connection refused |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-a4a3e |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-a4a3e |
InferenceServiceReady |
InferenceService [success-200-isvc-a4a3e] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-3af6d-55559fdf56 |
SuccessfulCreate |
Created pod: switch-graph-3af6d-55559fdf56-6zztd | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-3af6d |
ScalingReplicaSet |
Scaled up replica set switch-graph-3af6d-55559fdf56 from 0 to 1 | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-a4a3e |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-a4a3e |
InferenceServiceReady |
InferenceService [error-404-isvc-a4a3e] is Ready | |
kserve-ci-e2e-test |
kubelet |
switch-graph-3af6d-55559fdf56-6zztd |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:ca6150683b88d0bc774f8e42803fb5db7032d026ca2a7ea35068f270ce0317f1" already present on machine | |
kserve-ci-e2e-test |
kubelet |
switch-graph-3af6d-55559fdf56-6zztd |
Started |
Started container switch-graph-3af6d | |
kserve-ci-e2e-test |
kubelet |
switch-graph-3af6d-55559fdf56-6zztd |
Created |
Created container: switch-graph-3af6d | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-3af6d |
InferenceGraphReady |
InferenceGraph [switch-graph-3af6d] is Ready | |
kserve-ci-e2e-test |
multus |
switch-graph-3af6d-55559fdf56-6zztd |
AddedInterface |
Add eth0 [10.134.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-a4a3e |
ScalingReplicaSet |
Scaled up replica set splitter-graph-a4a3e-76cd97966c from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-a4a3e-76cd97966c-7r5tr |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "splitter-graph-a4a3e-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-a4a3e-76cd97966c |
SuccessfulCreate |
Created pod: splitter-graph-a4a3e-76cd97966c-7r5tr | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-a4a3e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-a4a3e |
InferenceGraphReady |
InferenceGraph [splitter-graph-a4a3e] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-a4a3e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
multus |
splitter-graph-a4a3e-76cd97966c-7r5tr |
AddedInterface |
Add eth0 [10.134.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-a4a3e-76cd97966c-7r5tr |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:ca6150683b88d0bc774f8e42803fb5db7032d026ca2a7ea35068f270ce0317f1" already present on machine | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-a4a3e-76cd97966c-7r5tr |
Created |
Created container: splitter-graph-a4a3e | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-a4a3e-76cd97966c-7r5tr |
Started |
Started container splitter-graph-a4a3e | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-3af6d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-3af6d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-3af6d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-3af6d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-a4a3e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-a4a3e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-a4a3e |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-a4a3e |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-3af6d |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-3af6d |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
splitter-graph-a4a3e-76cd97966c-7r5tr |
Killing |
Stopping container splitter-graph-a4a3e | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a4a3e-predictor-df7cf9c54-h6kkw |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a4a3e-predictor-6667499d67-jztfn |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.30:8643/healthz": dial tcp 10.133.0.30:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a4a3e-predictor-6667499d67-jztfn |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a4a3e-predictor-6667499d67-jztfn |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a4a3e-predictor-df7cf9c54-h6kkw |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a4a3e-predictor-df7cf9c54-h6kkw |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.44:8643/healthz": dial tcp 10.132.0.44:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-a4a3e-predictor-df7cf9c54-h6kkw |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.44:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-a4a3e-76cd97966c-7r5tr |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3af6d-predictor-67876986cd-bjgx2 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
switch-graph-3af6d-55559fdf56-6zztd |
Killing |
Stopping container switch-graph-3af6d | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-3af6d-predictor-67876986cd-bjgx2 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3af6d-predictor-7dd87fb4f9-zgxk4 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-3af6d-predictor-7dd87fb4f9-zgxk4 |
Killing |
Stopping container kube-rbac-proxy | |
| (x5) | kserve-ci-e2e-test |
kubelet |
switch-graph-3af6d-55559fdf56-6zztd |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |