| Time | Namespace | Component | RelatedObject | Reason | Message |
|---|---|---|---|---|---|
kserve-ci-e2e-test |
success-200-isvc-e8943-predictor-59544c7895-2k9ld |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-e8943-predictor-59544c7895-2k9ld to ip-10-0-128-233.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-91263-5f5cb88ccb-cqznj |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-91263-5f5cb88ccb-cqznj to ip-10-0-134-97.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-35019-predictor-6647d995d5-4jxvp |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-35019-predictor-6647d995d5-4jxvp to ip-10-0-128-233.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-43619-predictor-857f7657f5-b28dc |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-43619-predictor-857f7657f5-b28dc to ip-10-0-128-233.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-43619-cb57bcfb7-mjrs4 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-43619-cb57bcfb7-mjrs4 to ip-10-0-134-97.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-cbd18-predictor-69b7b99dcb-lxn88 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-cbd18-predictor-69b7b99dcb-lxn88 to ip-10-0-128-233.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-91263-predictor-5f479d8546-d2nfn |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-91263-predictor-5f479d8546-d2nfn to ip-10-0-128-233.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-02ff4-854c9cfd46-slghq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-02ff4-854c9cfd46-slghq to ip-10-0-134-97.ec2.internal | ||
kserve-ci-e2e-test |
model-chainer-659b76d949-vdqbw |
Scheduled |
Successfully assigned kserve-ci-e2e-test/model-chainer-659b76d949-vdqbw to ip-10-0-134-97.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-db27e-predictor-649d5848b4-qx64q |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-db27e-predictor-649d5848b4-qx64q to ip-10-0-128-233.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-e8943-predictor-6c55cf5cbb-d4pj5 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-e8943-predictor-6c55cf5cbb-d4pj5 to ip-10-0-128-233.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd to ip-10-0-128-52.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-492f6-predictor-598d856bcd-kxgf8 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-492f6-predictor-598d856bcd-kxgf8 to ip-10-0-128-233.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-5859e-7c56564db7-krtf4 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-5859e-7c56564db7-krtf4 to ip-10-0-134-97.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-e8943-86d8b689d5-jprd6 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-e8943-86d8b689d5-jprd6 to ip-10-0-134-97.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-cbd18-7b6659b66b-s74jh |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-cbd18-7b6659b66b-s74jh to ip-10-0-134-97.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-db27e-predictor-6c64cf5797-l4fz2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-db27e-predictor-6c64cf5797-l4fz2 to ip-10-0-128-233.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-575ad-64d456d77c-rsdgn |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-575ad-64d456d77c-rsdgn to ip-10-0-134-97.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-db27e-7896bb48fc-nhrvq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-db27e-7896bb48fc-nhrvq to ip-10-0-134-97.ec2.internal | ||
kserve-ci-e2e-test |
isvc-xgboost-graph-predictor-555fc69d56-xcfvp |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp to ip-10-0-128-52.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-02ff4-predictor-7748bf4588-2k6pk |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-02ff4-predictor-7748bf4588-2k6pk to ip-10-0-128-233.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-492f6-predictor-6698d5fcc5-m8sgw |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-492f6-predictor-6698d5fcc5-m8sgw to ip-10-0-128-233.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-5859e-predictor-55f99fd8b8-5mg6d |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-5859e-predictor-55f99fd8b8-5mg6d to ip-10-0-128-233.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-43619-predictor-8695456798-vdxlr |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-43619-predictor-8695456798-vdxlr to ip-10-0-128-233.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-cbd18-predictor-749f5f5477-xmgml |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-cbd18-predictor-749f5f5477-xmgml to ip-10-0-128-233.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-492f6-67c988957c-hcprv |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-492f6-67c988957c-hcprv to ip-10-0-134-97.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-575ad-predictor-7d77b767d4-xbql6 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-575ad-predictor-7d77b767d4-xbql6 to ip-10-0-128-233.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-91263-predictor-867c654bd7-l96fd |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-91263-predictor-867c654bd7-l96fd to ip-10-0-128-233.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-35019-predictor-86b8b845bd-2l9hx |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-35019-predictor-86b8b845bd-2l9hx to ip-10-0-128-233.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-5859e-predictor-76c758fc77-sq8sr |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-5859e-predictor-76c758fc77-sq8sr to ip-10-0-128-233.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv to ip-10-0-128-52.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-35019-8565bdbbb7-fcg64 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-35019-8565bdbbb7-fcg64 to ip-10-0-134-97.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-02ff4-predictor-6d45dbc5c7-j66tq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-02ff4-predictor-6d45dbc5c7-j66tq to ip-10-0-128-233.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-575ad-predictor-9f95b5ddf-p6vcw |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-575ad-predictor-9f95b5ddf-p6vcw to ip-10-0-128-233.ec2.internal | ||
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "isvc-sklearn-graph-1": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-cbd18-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-cbd18-predictor-749f5f5477 from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
UpdateFailed |
Failed to update status for InferenceService "isvc-sklearn-graph-1": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "isvc-sklearn-graph-1": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-1-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-1-predictor-6449ff9c59 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-1-predictor-6449ff9c59 |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:10d11eb5432cd26f2fc3d3776649a96acdfb9765433f51db6bb597849bf71311" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-cbd18-predictor-749f5f5477-xmgml |
Pulling |
Pulling image "quay.io/opendatahub/success-200-isvc:odh-pr-1733" | |
kserve-ci-e2e-test |
multus |
success-200-isvc-cbd18-predictor-749f5f5477-xmgml |
AddedInterface |
Add eth0 [10.134.0.19/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-2-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-2-predictor-7894f545fc from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xcfvp |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "isvc-xgboost-graph-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-2-predictor-7894f545fc |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv |
AddedInterface |
Add eth0 [10.133.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-cbd18-predictor-749f5f5477 |
SuccessfulCreate |
Created pod: success-200-isvc-cbd18-predictor-749f5f5477-xmgml | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd |
AddedInterface |
Add eth0 [10.133.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-xgboost-graph-predictor-555fc69d56 |
SuccessfulCreate |
Created pod: isvc-xgboost-graph-predictor-555fc69d56-xcfvp | |
kserve-ci-e2e-test |
deployment-controller |
isvc-xgboost-graph-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-xgboost-graph-predictor-555fc69d56 from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-cbd18-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-cbd18-predictor-69b7b99dcb from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-cbd18-predictor-69b7b99dcb |
SuccessfulCreate |
Created pod: error-404-isvc-cbd18-predictor-69b7b99dcb-lxn88 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-cbd18-predictor-69b7b99dcb-lxn88 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-cbd18-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:10d11eb5432cd26f2fc3d3776649a96acdfb9765433f51db6bb597849bf71311" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xcfvp |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xcfvp |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xcfvp |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:10d11eb5432cd26f2fc3d3776649a96acdfb9765433f51db6bb597849bf71311" already present on machine | |
kserve-ci-e2e-test |
multus |
isvc-xgboost-graph-predictor-555fc69d56-xcfvp |
AddedInterface |
Add eth0 [10.133.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
multus |
error-404-isvc-cbd18-predictor-69b7b99dcb-lxn88 |
AddedInterface |
Add eth0 [10.134.0.20/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-cbd18-predictor-69b7b99dcb-lxn88 |
Pulling |
Pulling image "quay.io/opendatahub/error-404-isvc:odh-pr-1733" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1733" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1733" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xcfvp |
Pulling |
Pulling image "kserve/xgbserver:latest" | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-cbd18-predictor-749f5f5477-xmgml |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-cbd18-predictor-69b7b99dcb-lxn88 |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-cbd18-predictor-69b7b99dcb-lxn88 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-cbd18-predictor-69b7b99dcb-lxn88 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-cbd18-predictor-69b7b99dcb-lxn88 |
Pulled |
Successfully pulled image "quay.io/opendatahub/error-404-isvc:odh-pr-1733" in 14.864s (14.864s including waiting). Image size: 1334693997 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-cbd18-predictor-749f5f5477-xmgml |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-cbd18-predictor-749f5f5477-xmgml |
Pulled |
Successfully pulled image "quay.io/opendatahub/success-200-isvc:odh-pr-1733" in 15.677s (15.677s including waiting). Image size: 1335613554 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-cbd18-predictor-749f5f5477-xmgml |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-cbd18-predictor-69b7b99dcb-lxn88 |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.651s (2.651s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-cbd18-predictor-749f5f5477-xmgml |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.679s (2.679s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-cbd18-predictor-69b7b99dcb-lxn88 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-cbd18-predictor-749f5f5477-xmgml |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-cbd18-predictor-69b7b99dcb-lxn88 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-cbd18-predictor-749f5f5477-xmgml |
Created |
Created container: kube-rbac-proxy | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-cbd18-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-cbd18-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xcfvp |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1733" in 28.177s (28.177s including waiting). Image size: 1555805621 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xcfvp |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xcfvp |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xcfvp |
Pulled |
Successfully pulled image "kserve/xgbserver:latest" in 27.471s (27.471s including waiting). Image size: 1305567991 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1733" in 28.901s (28.901s including waiting). Image size: 1555805621 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.832s (2.832s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.064s (2.064s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xcfvp |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.519s (2.519s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xcfvp |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xcfvp |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd |
Started |
Started container kube-rbac-proxy | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-cbd18-predictor-749f5f5477-xmgml |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.19:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-cbd18-predictor-69b7b99dcb-lxn88 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.20:8080: connect: connection refused |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-cbd18-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-cbd18-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-cbd18 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-cbd18 |
InferenceServiceReady |
InferenceService [success-200-isvc-cbd18] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-cbd18 |
InferenceServiceReady |
InferenceService [error-404-isvc-cbd18] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-cbd18 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-cbd18-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-cbd18-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-cbd18-7b6659b66b |
SuccessfulCreate |
Created pod: switch-graph-cbd18-7b6659b66b-s74jh | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-cbd18 |
ScalingReplicaSet |
Scaled up replica set switch-graph-cbd18-7b6659b66b from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-cbd18 |
InferenceGraphReady |
InferenceGraph [switch-graph-cbd18] is Ready | |
kserve-ci-e2e-test |
kubelet |
switch-graph-cbd18-7b6659b66b-s74jh |
Pulling |
Pulling image "quay.io/opendatahub/kserve-router@sha256:8345cd1e0b8ad942d68e2fe65c99d692fac6452f8ea219efb6700aaf29b3e2fa" | |
kserve-ci-e2e-test |
multus |
switch-graph-cbd18-7b6659b66b-s74jh |
AddedInterface |
Add eth0 [10.132.0.25/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-cbd18 |
UpdateFailed |
Failed to update status for InferenceGraph "switch-graph-cbd18": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "switch-graph-cbd18": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-cbd18 |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "switch-graph-cbd18": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
switch-graph-cbd18-7b6659b66b-s74jh |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-cbd18-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
switch-graph-cbd18-7b6659b66b-s74jh |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-router@sha256:8345cd1e0b8ad942d68e2fe65c99d692fac6452f8ea219efb6700aaf29b3e2fa" in 2.599s (2.599s including waiting). Image size: 218072700 bytes. | |
kserve-ci-e2e-test |
kubelet |
switch-graph-cbd18-7b6659b66b-s74jh |
Started |
Started container switch-graph-cbd18 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-cbd18-7b6659b66b-s74jh |
Created |
Created container: switch-graph-cbd18 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-91263-predictor-5f479d8546-d2nfn |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-91263-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
switch-graph-cbd18-7b6659b66b-s74jh |
Killing |
Stopping container switch-graph-cbd18 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-91263-predictor-867c654bd7-l96fd |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1733" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-91263-predictor-867c654bd7-l96fd |
Created |
Created container: kserve-container | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-cbd18-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-91263-predictor-867c654bd7-l96fd |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-cbd18-predictor-749f5f5477-xmgml |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.19:8643/healthz": dial tcp 10.134.0.19:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-91263-predictor-867c654bd7-l96fd |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-cbd18-predictor-749f5f5477-xmgml |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-cbd18-predictor-749f5f5477-xmgml |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-91263-predictor-867c654bd7-l96fd |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-91263 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-91263": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-91263 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-91263": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-91263": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-91263-predictor-867c654bd7-l96fd |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-91263-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-91263-predictor-5f479d8546 from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-91263 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-91263": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-91263 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-91263": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-91263": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-91263-predictor-5f479d8546 |
SuccessfulCreate |
Created pod: error-404-isvc-91263-predictor-5f479d8546-d2nfn | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-cbd18-predictor-69b7b99dcb-lxn88 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-cbd18-predictor-69b7b99dcb-lxn88 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-91263-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-91263-predictor-867c654bd7 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-91263-predictor-867c654bd7 |
SuccessfulCreate |
Created pod: success-200-isvc-91263-predictor-867c654bd7-l96fd | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-cbd18-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
multus |
success-200-isvc-91263-predictor-867c654bd7-l96fd |
AddedInterface |
Add eth0 [10.134.0.21/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-91263-predictor-5f479d8546-d2nfn |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1733" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-91263-predictor-5f479d8546-d2nfn |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-91263-predictor-5f479d8546-d2nfn |
AddedInterface |
Add eth0 [10.134.0.22/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-cbd18-predictor-69b7b99dcb-lxn88 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.20:8643/healthz": dial tcp 10.134.0.20:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-91263-predictor-5f479d8546-d2nfn |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-91263-predictor-5f479d8546-d2nfn |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-91263-predictor-5f479d8546-d2nfn |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-91263-predictor-5f479d8546-d2nfn |
Created |
Created container: kube-rbac-proxy | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
InferenceServiceReady |
InferenceService [isvc-xgboost-graph] is Ready | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-91263-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-91263-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-91263-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-91263-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-1] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-2] is Ready | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-cbd18-7b6659b66b-s74jh |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-91263-predictor-867c654bd7-l96fd |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.21:8080: connect: connection refused |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-91263 |
InferenceServiceReady |
InferenceService [error-404-isvc-91263] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-91263 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-91263 |
InferenceServiceReady |
InferenceService [success-200-isvc-91263] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-91263 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
InferenceGraphController |
model-chainer |
InferenceGraphReady |
InferenceGraph [model-chainer] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
model-chainer |
ScalingReplicaSet |
Scaled up replica set model-chainer-659b76d949 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
model-chainer-659b76d949 |
SuccessfulCreate |
Created pod: model-chainer-659b76d949-vdqbw | |
kserve-ci-e2e-test |
kubelet |
model-chainer-659b76d949-vdqbw |
Started |
Started container model-chainer | |
kserve-ci-e2e-test |
kubelet |
model-chainer-659b76d949-vdqbw |
Created |
Created container: model-chainer | |
kserve-ci-e2e-test |
kubelet |
model-chainer-659b76d949-vdqbw |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:8345cd1e0b8ad942d68e2fe65c99d692fac6452f8ea219efb6700aaf29b3e2fa" already present on machine | |
kserve-ci-e2e-test |
multus |
model-chainer-659b76d949-vdqbw |
AddedInterface |
Add eth0 [10.132.0.26/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
model-chainer-659b76d949-vdqbw |
Killing |
Stopping container model-chainer | |
kserve-ci-e2e-test |
kubelet |
switch-graph-91263-5f5cb88ccb-cqznj |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:8345cd1e0b8ad942d68e2fe65c99d692fac6452f8ea219efb6700aaf29b3e2fa" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-43619 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-43619": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-43619": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-91263-5f5cb88ccb |
SuccessfulCreate |
Created pod: switch-graph-91263-5f5cb88ccb-cqznj | |
kserve-ci-e2e-test |
kubelet |
switch-graph-91263-5f5cb88ccb-cqznj |
Started |
Started container switch-graph-91263 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-91263-5f5cb88ccb-cqznj |
Created |
Created container: switch-graph-91263 | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-43619-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-43619-predictor-857f7657f5 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-43619-predictor-857f7657f5 |
SuccessfulCreate |
Created pod: success-200-isvc-43619-predictor-857f7657f5-b28dc | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-91263 |
ScalingReplicaSet |
Scaled up replica set switch-graph-91263-5f5cb88ccb from 0 to 1 | |
kserve-ci-e2e-test |
multus |
switch-graph-91263-5f5cb88ccb-cqznj |
AddedInterface |
Add eth0 [10.132.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-43619 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-43619": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-43619 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-43619": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-43619 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-43619": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-43619": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xcfvp |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xcfvp |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-43619-predictor-857f7657f5-b28dc |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-43619-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-43619-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-43619-predictor-8695456798 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-43619-predictor-8695456798-vdxlr |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-43619-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-43619-predictor-8695456798 |
SuccessfulCreate |
Created pod: error-404-isvc-43619-predictor-8695456798-vdxlr | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-91263 |
InferenceGraphReady |
InferenceGraph [switch-graph-91263] is Ready | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-43619-predictor-857f7657f5-b28dc |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-43619-predictor-8695456798-vdxlr |
AddedInterface |
Add eth0 [10.134.0.24/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-43619-predictor-8695456798-vdxlr |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-43619-predictor-8695456798-vdxlr |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-43619-predictor-8695456798-vdxlr |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-43619-predictor-8695456798-vdxlr |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-43619-predictor-8695456798-vdxlr |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1733" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-43619-predictor-8695456798-vdxlr |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-43619-predictor-857f7657f5-b28dc |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-43619-predictor-857f7657f5-b28dc |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-43619-predictor-857f7657f5-b28dc |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-43619-predictor-857f7657f5-b28dc |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1733" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-43619-predictor-857f7657f5-b28dc |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-43619-predictor-857f7657f5-b28dc |
AddedInterface |
Add eth0 [10.134.0.23/23] from ovn-kubernetes | |
| (x10) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.32:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.32:8643/healthz": dial tcp 10.133.0.32:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.30:8643/healthz": dial tcp 10.133.0.30:8643: connect: connection refused | |
| (x10) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.30:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xcfvp |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.31:8643/healthz": dial tcp 10.133.0.31:8643: connect: connection refused | |
| (x8) | kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xcfvp |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.31:8080: connect: connection refused |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-91263-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-91263-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-91263-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-91263-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
model-chainer-659b76d949-vdqbw |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-91263 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-43619-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-43619-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-43619-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-91263 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-43619-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-43619-predictor-857f7657f5-b28dc |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.23:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-43619-predictor-8695456798-vdxlr |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.24:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-43619 |
InferenceServiceReady |
InferenceService [success-200-isvc-43619] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-43619 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-43619 |
InferenceServiceReady |
InferenceService [error-404-isvc-43619] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-43619 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
kubelet |
sequence-graph-43619-cb57bcfb7-mjrs4 |
Created |
Created container: sequence-graph-43619 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-43619-cb57bcfb7-mjrs4 |
Started |
Started container sequence-graph-43619 | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-43619 |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "sequence-graph-43619": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-43619 |
UpdateFailed |
Failed to update status for InferenceGraph "sequence-graph-43619": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "sequence-graph-43619": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-43619 |
InferenceGraphReady |
InferenceGraph [sequence-graph-43619] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-43619 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-43619-cb57bcfb7 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
sequence-graph-43619-cb57bcfb7-mjrs4 |
AddedInterface |
Add eth0 [10.132.0.28/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-43619-cb57bcfb7 |
SuccessfulCreate |
Created pod: sequence-graph-43619-cb57bcfb7-mjrs4 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-43619-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
sequence-graph-43619-cb57bcfb7-mjrs4 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:8345cd1e0b8ad942d68e2fe65c99d692fac6452f8ea219efb6700aaf29b3e2fa" already present on machine | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-43619-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-43619-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-43619 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-43619-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-43619 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-43619 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-43619 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-91263-predictor-5f479d8546-d2nfn |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-02ff4-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-02ff4-predictor-6d45dbc5c7 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-02ff4-predictor-7748bf4588 |
SuccessfulCreate |
Created pod: error-404-isvc-02ff4-predictor-7748bf4588-2k6pk | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-02ff4-predictor-6d45dbc5c7-j66tq |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-02ff4-predictor-6d45dbc5c7-j66tq |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-02ff4-predictor-6d45dbc5c7-j66tq |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-02ff4-predictor-6d45dbc5c7-j66tq |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
switch-graph-91263-5f5cb88ccb-cqznj |
Killing |
Stopping container switch-graph-91263 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-02ff4-predictor-6d45dbc5c7-j66tq |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1733" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-02ff4-predictor-6d45dbc5c7-j66tq |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-02ff4-predictor-6d45dbc5c7-j66tq |
AddedInterface |
Add eth0 [10.134.0.25/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-02ff4-predictor-6d45dbc5c7 |
SuccessfulCreate |
Created pod: success-200-isvc-02ff4-predictor-6d45dbc5c7-j66tq | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-02ff4-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-02ff4-predictor-7748bf4588 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-02ff4-predictor-7748bf4588-2k6pk |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-02ff4-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-91263-predictor-867c654bd7-l96fd |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-91263-predictor-867c654bd7-l96fd |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-02ff4 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-02ff4": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-02ff4 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-02ff4": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-02ff4 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-02ff4": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-02ff4": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-02ff4 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-02ff4": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-02ff4": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-91263-predictor-5f479d8546-d2nfn |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-02ff4-predictor-7748bf4588-2k6pk |
AddedInterface |
Add eth0 [10.134.0.26/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-02ff4-predictor-7748bf4588-2k6pk |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1733" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-02ff4-predictor-7748bf4588-2k6pk |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-02ff4-predictor-7748bf4588-2k6pk |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-02ff4-predictor-7748bf4588-2k6pk |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-02ff4-predictor-7748bf4588-2k6pk |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-02ff4-predictor-7748bf4588-2k6pk |
Started |
Started container kube-rbac-proxy | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-91263-predictor-5f479d8546-d2nfn |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.22:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-91263-predictor-5f479d8546-d2nfn |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.22:8643/healthz": dial tcp 10.134.0.22:8643: connect: connection refused | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-02ff4-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-02ff4-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-02ff4-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-02ff4-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-91263-5f5cb88ccb-cqznj |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-02ff4-predictor-6d45dbc5c7-j66tq |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.25:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-02ff4-predictor-7748bf4588-2k6pk |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.26:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-02ff4 |
InferenceServiceReady |
InferenceService [success-200-isvc-02ff4] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-02ff4 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-02ff4 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-02ff4 |
InferenceServiceReady |
InferenceService [error-404-isvc-02ff4] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e8943 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-e8943": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-e8943-predictor-59544c7895 |
SuccessfulCreate |
Created pod: success-200-isvc-e8943-predictor-59544c7895-2k9ld | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e8943 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-e8943": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-e8943": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-e8943-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-e8943-predictor-59544c7895 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-43619-predictor-857f7657f5-b28dc |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-43619-cb57bcfb7-mjrs4 |
Killing |
Stopping container sequence-graph-43619 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-43619-predictor-857f7657f5-b28dc |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e8943-predictor-6c55cf5cbb-d4pj5 |
Started |
Started container kube-rbac-proxy | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-02ff4-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
multus |
error-404-isvc-e8943-predictor-6c55cf5cbb-d4pj5 |
AddedInterface |
Add eth0 [10.134.0.28/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e8943-predictor-59544c7895-2k9ld |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e8943-predictor-59544c7895-2k9ld |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1733" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-43619-predictor-8695456798-vdxlr |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-43619-predictor-8695456798-vdxlr |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e8943-predictor-59544c7895-2k9ld |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e8943-predictor-59544c7895-2k9ld |
Started |
Started container kube-rbac-proxy | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-02ff4-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-02ff4-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
multus |
success-200-isvc-e8943-predictor-59544c7895-2k9ld |
AddedInterface |
Add eth0 [10.134.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e8943-predictor-6c55cf5cbb-d4pj5 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1733" already present on machine | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-e8943-predictor-6c55cf5cbb |
SuccessfulCreate |
Created pod: error-404-isvc-e8943-predictor-6c55cf5cbb-d4pj5 | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-02ff4-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e8943-predictor-59544c7895-2k9ld |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-02ff4-854c9cfd46 |
SuccessfulCreate |
Created pod: ensemble-graph-02ff4-854c9cfd46-slghq | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e8943-predictor-59544c7895-2k9ld |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-e8943-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-e8943-predictor-6c55cf5cbb from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e8943-predictor-6c55cf5cbb-d4pj5 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e8943-predictor-6c55cf5cbb-d4pj5 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e8943 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-e8943": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-e8943": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e8943-predictor-6c55cf5cbb-d4pj5 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e8943-predictor-6c55cf5cbb-d4pj5 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e8943 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-e8943": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-02ff4 |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-02ff4-854c9cfd46 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-02ff4 |
InferenceGraphReady |
InferenceGraph [ensemble-graph-02ff4] is Ready | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-02ff4-854c9cfd46-slghq |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "ensemble-graph-02ff4-serving-cert" not found | |
kserve-ci-e2e-test |
multus |
ensemble-graph-02ff4-854c9cfd46-slghq |
AddedInterface |
Add eth0 [10.132.0.29/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-02ff4-854c9cfd46-slghq |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:8345cd1e0b8ad942d68e2fe65c99d692fac6452f8ea219efb6700aaf29b3e2fa" already present on machine | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-02ff4-854c9cfd46-slghq |
Started |
Started container ensemble-graph-02ff4 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-02ff4-854c9cfd46-slghq |
Created |
Created container: ensemble-graph-02ff4 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-02ff4-predictor-6d45dbc5c7-j66tq |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-02ff4-854c9cfd46-slghq |
Killing |
Stopping container ensemble-graph-02ff4 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-db27e-predictor-6c64cf5797-l4fz2 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-db27e-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-db27e |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-db27e": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-db27e |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-db27e": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-db27e": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-db27e |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-db27e": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-db27e": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-db27e |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-db27e": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-db27e-predictor-649d5848b4-qx64q |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-db27e-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-db27e-predictor-6c64cf5797 |
SuccessfulCreate |
Created pod: success-200-isvc-db27e-predictor-6c64cf5797-l4fz2 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-02ff4-predictor-7748bf4588-2k6pk |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-02ff4-predictor-7748bf4588-2k6pk |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-02ff4-predictor-6d45dbc5c7-j66tq |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-db27e-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-db27e-predictor-6c64cf5797 from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-db27e-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-db27e-predictor-649d5848b4 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-db27e-predictor-649d5848b4 |
SuccessfulCreate |
Created pod: error-404-isvc-db27e-predictor-649d5848b4-qx64q | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-db27e-predictor-6c64cf5797-l4fz2 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-db27e-predictor-6c64cf5797-l4fz2 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-db27e-predictor-6c64cf5797-l4fz2 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-db27e-predictor-649d5848b4-qx64q |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-db27e-predictor-6c64cf5797-l4fz2 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-db27e-predictor-6c64cf5797-l4fz2 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1733" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-db27e-predictor-649d5848b4-qx64q |
AddedInterface |
Add eth0 [10.134.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-db27e-predictor-649d5848b4-qx64q |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-db27e-predictor-649d5848b4-qx64q |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1733" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-db27e-predictor-6c64cf5797-l4fz2 |
AddedInterface |
Add eth0 [10.134.0.29/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-db27e-predictor-649d5848b4-qx64q |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-db27e-predictor-6c64cf5797-l4fz2 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-db27e-predictor-649d5848b4-qx64q |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-db27e-predictor-649d5848b4-qx64q |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-02ff4-predictor-6d45dbc5c7-j66tq |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.25:8643/healthz": dial tcp 10.134.0.25:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-02ff4-predictor-7748bf4588-2k6pk |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.26:8643/healthz": dial tcp 10.134.0.26:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-43619-cb57bcfb7-mjrs4 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e8943-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e8943-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-e8943-predictor-59544c7895-2k9ld |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.27:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-e8943-predictor-6c55cf5cbb-d4pj5 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.28:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-02ff4-854c9cfd46-slghq |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-db27e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-db27e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-db27e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-db27e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e8943-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e8943-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e8943 |
InferenceServiceReady |
InferenceService [error-404-isvc-e8943] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e8943 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e8943 |
InferenceServiceReady |
InferenceService [success-200-isvc-e8943] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e8943 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-db27e |
InferenceServiceReady |
InferenceService [success-200-isvc-db27e] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-db27e |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-db27e |
InferenceServiceReady |
InferenceService [error-404-isvc-db27e] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-db27e |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
multus |
sequence-graph-e8943-86d8b689d5-jprd6 |
AddedInterface |
Add eth0 [10.132.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-e8943-86d8b689d5-jprd6 |
Started |
Started container sequence-graph-e8943 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e8943-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e8943-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-e8943 |
InferenceGraphReady |
InferenceGraph [sequence-graph-e8943] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-e8943 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-e8943-86d8b689d5 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-e8943-86d8b689d5 |
SuccessfulCreate |
Created pod: sequence-graph-e8943-86d8b689d5-jprd6 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-e8943-86d8b689d5-jprd6 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:8345cd1e0b8ad942d68e2fe65c99d692fac6452f8ea219efb6700aaf29b3e2fa" already present on machine | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-e8943-86d8b689d5-jprd6 |
Created |
Created container: sequence-graph-e8943 | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e8943-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e8943-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5859e-predictor-55f99fd8b8-5mg6d |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e8943-predictor-59544c7895-2k9ld |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-e8943-86d8b689d5-jprd6 |
Killing |
Stopping container sequence-graph-e8943 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-5859e-predictor-76c758fc77 |
SuccessfulCreate |
Created pod: success-200-isvc-5859e-predictor-76c758fc77-sq8sr | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e8943-predictor-6c55cf5cbb-d4pj5 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-5859e |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-5859e": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-5859e": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5859e-predictor-76c758fc77-sq8sr |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5859e-predictor-76c758fc77-sq8sr |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-5859e |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-5859e": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5859e-predictor-55f99fd8b8-5mg6d |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e8943-predictor-59544c7895-2k9ld |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5859e-predictor-55f99fd8b8-5mg6d |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1733" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-5859e-predictor-55f99fd8b8-5mg6d |
AddedInterface |
Add eth0 [10.134.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-5859e-predictor-55f99fd8b8 |
SuccessfulCreate |
Created pod: error-404-isvc-5859e-predictor-55f99fd8b8-5mg6d | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5859e-predictor-76c758fc77-sq8sr |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-5859e-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-5859e-predictor-76c758fc77 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-db27e |
InferenceGraphReady |
InferenceGraph [ensemble-graph-db27e] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-db27e |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-db27e-7896bb48fc from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5859e-predictor-76c758fc77-sq8sr |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5859e-predictor-76c758fc77-sq8sr |
Created |
Created container: kserve-container | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-db27e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-db27e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-5859e |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-5859e": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
multus |
success-200-isvc-5859e-predictor-76c758fc77-sq8sr |
AddedInterface |
Add eth0 [10.134.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5859e-predictor-76c758fc77-sq8sr |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1733" already present on machine | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-5859e |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-5859e": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-5859e": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-db27e-7896bb48fc-nhrvq |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "ensemble-graph-db27e-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-db27e-7896bb48fc |
SuccessfulCreate |
Created pod: ensemble-graph-db27e-7896bb48fc-nhrvq | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-5859e-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-5859e-predictor-55f99fd8b8 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5859e-predictor-55f99fd8b8-5mg6d |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e8943-predictor-6c55cf5cbb-d4pj5 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
multus |
ensemble-graph-db27e-7896bb48fc-nhrvq |
AddedInterface |
Add eth0 [10.132.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-db27e-7896bb48fc-nhrvq |
Started |
Started container ensemble-graph-db27e | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5859e-predictor-55f99fd8b8-5mg6d |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-db27e-7896bb48fc-nhrvq |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:8345cd1e0b8ad942d68e2fe65c99d692fac6452f8ea219efb6700aaf29b3e2fa" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5859e-predictor-55f99fd8b8-5mg6d |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-db27e-7896bb48fc-nhrvq |
Created |
Created container: ensemble-graph-db27e | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e8943-predictor-59544c7895-2k9ld |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.27:8643/healthz": dial tcp 10.134.0.27:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e8943-predictor-6c55cf5cbb-d4pj5 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.28:8643/healthz": dial tcp 10.134.0.28:8643: connect: connection refused | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-db27e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-db27e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-e8943-86d8b689d5-jprd6 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-5859e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-5859e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-db27e |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-db27e |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-5859e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-5859e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-5859e |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-5859e |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-5859e |
InferenceServiceReady |
InferenceService [success-200-isvc-5859e] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-5859e |
InferenceServiceReady |
InferenceService [error-404-isvc-5859e] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-5859e-7c56564db7 |
SuccessfulCreate |
Created pod: sequence-graph-5859e-7c56564db7-krtf4 | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-5859e |
InferenceGraphReady |
InferenceGraph [sequence-graph-5859e] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-5859e |
ScalingReplicaSet |
Scaled up replica set sequence-graph-5859e-7c56564db7 from 0 to 1 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-5859e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
sequence-graph-5859e-7c56564db7-krtf4 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:8345cd1e0b8ad942d68e2fe65c99d692fac6452f8ea219efb6700aaf29b3e2fa" already present on machine | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-5859e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
sequence-graph-5859e-7c56564db7-krtf4 |
Started |
Started container sequence-graph-5859e | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-5859e-7c56564db7-krtf4 |
Created |
Created container: sequence-graph-5859e | |
kserve-ci-e2e-test |
multus |
sequence-graph-5859e-7c56564db7-krtf4 |
AddedInterface |
Add eth0 [10.132.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-5859e |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-5859e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-5859e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-5859e |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-5859e |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-5859e |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-35019-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-35019-predictor-6647d995d5 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-35019-predictor-6647d995d5 |
SuccessfulCreate |
Created pod: success-200-isvc-35019-predictor-6647d995d5-4jxvp | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-35019-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-35019-predictor-86b8b845bd from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-35019-predictor-86b8b845bd |
SuccessfulCreate |
Created pod: error-404-isvc-35019-predictor-86b8b845bd-2l9hx | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-db27e-predictor-649d5848b4-qx64q |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-db27e-predictor-649d5848b4-qx64q |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-db27e-7896bb48fc-nhrvq |
Killing |
Stopping container ensemble-graph-db27e | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-35019 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-35019": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-35019": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-35019 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-35019": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-35019 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-35019": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-35019": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-db27e-predictor-6c64cf5797-l4fz2 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-db27e-predictor-6c64cf5797-l4fz2 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-35019 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-35019": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-35019-predictor-6647d995d5-4jxvp |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-35019-predictor-86b8b845bd-2l9hx |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-35019-predictor-6647d995d5-4jxvp |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-35019-predictor-86b8b845bd-2l9hx |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-35019-predictor-86b8b845bd-2l9hx |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-35019-predictor-6647d995d5-4jxvp |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-35019-predictor-6647d995d5-4jxvp |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-35019-predictor-86b8b845bd-2l9hx |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-35019-predictor-86b8b845bd-2l9hx |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-35019-predictor-86b8b845bd-2l9hx |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1733" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-35019-predictor-86b8b845bd-2l9hx |
AddedInterface |
Add eth0 [10.134.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
success-200-isvc-35019-predictor-6647d995d5-4jxvp |
AddedInterface |
Add eth0 [10.134.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-35019-predictor-6647d995d5-4jxvp |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1733" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-35019-predictor-6647d995d5-4jxvp |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-db27e-predictor-649d5848b4-qx64q |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.30:8643/healthz": dial tcp 10.134.0.30:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-db27e-predictor-6c64cf5797-l4fz2 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.29:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-db27e-predictor-6c64cf5797-l4fz2 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.29:8643/healthz": dial tcp 10.134.0.29:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-db27e-predictor-649d5848b4-qx64q |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.30:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-db27e-7896bb48fc-nhrvq |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-35019-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-35019-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-35019-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-35019-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-35019-predictor-86b8b845bd-2l9hx |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.34:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-35019-predictor-6647d995d5-4jxvp |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.33:8080: connect: connection refused |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-35019 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-35019 |
InferenceServiceReady |
InferenceService [error-404-isvc-35019] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-35019 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-35019 |
InferenceServiceReady |
InferenceService [success-200-isvc-35019] is Ready | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-5859e-7c56564db7-krtf4 |
Killing |
Stopping container sequence-graph-5859e | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-575ad-predictor-7d77b767d4-xbql6 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-575ad-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-35019 |
InferenceGraphReady |
InferenceGraph [splitter-graph-35019] is Ready | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5859e-predictor-55f99fd8b8-5mg6d |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-575ad-predictor-9f95b5ddf |
SuccessfulCreate |
Created pod: success-200-isvc-575ad-predictor-9f95b5ddf-p6vcw | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-575ad-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-575ad-predictor-9f95b5ddf from 0 to 1 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-35019-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-35019-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5859e-predictor-55f99fd8b8-5mg6d |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-575ad |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-575ad": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-575ad |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-575ad": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-575ad": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-575ad-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-575ad-predictor-7d77b767d4 from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-575ad |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-575ad": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-575ad": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-575ad |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-575ad": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-575ad-predictor-7d77b767d4 |
SuccessfulCreate |
Created pod: error-404-isvc-575ad-predictor-7d77b767d4-xbql6 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5859e-predictor-76c758fc77-sq8sr |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5859e-predictor-76c758fc77-sq8sr |
Killing |
Stopping container kserve-container | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-35019-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-35019-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-35019-8565bdbbb7 |
SuccessfulCreate |
Created pod: splitter-graph-35019-8565bdbbb7-fcg64 | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-35019 |
ScalingReplicaSet |
Scaled up replica set splitter-graph-35019-8565bdbbb7 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-575ad-predictor-9f95b5ddf-p6vcw |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-575ad-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-35019-8565bdbbb7-fcg64 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:8345cd1e0b8ad942d68e2fe65c99d692fac6452f8ea219efb6700aaf29b3e2fa" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-575ad-predictor-7d77b767d4-xbql6 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-575ad-predictor-9f95b5ddf-p6vcw |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-575ad-predictor-9f95b5ddf-p6vcw |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
success-200-isvc-575ad-predictor-9f95b5ddf-p6vcw |
AddedInterface |
Add eth0 [10.134.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-575ad-predictor-9f95b5ddf-p6vcw |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-575ad-predictor-9f95b5ddf-p6vcw |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1733" already present on machine | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-35019-8565bdbbb7-fcg64 |
Started |
Started container splitter-graph-35019 | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-35019-8565bdbbb7-fcg64 |
Created |
Created container: splitter-graph-35019 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-575ad-predictor-7d77b767d4-xbql6 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
multus |
splitter-graph-35019-8565bdbbb7-fcg64 |
AddedInterface |
Add eth0 [10.132.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-575ad-predictor-9f95b5ddf-p6vcw |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-575ad-predictor-7d77b767d4-xbql6 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-575ad-predictor-7d77b767d4-xbql6 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-575ad-predictor-9f95b5ddf-p6vcw |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-575ad-predictor-7d77b767d4-xbql6 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-575ad-predictor-7d77b767d4-xbql6 |
AddedInterface |
Add eth0 [10.134.0.36/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-575ad-predictor-7d77b767d4-xbql6 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1733" already present on machine | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-5859e-predictor-76c758fc77-sq8sr |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.31:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-5859e-predictor-55f99fd8b8-5mg6d |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.32:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5859e-predictor-55f99fd8b8-5mg6d |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.32:8643/healthz": dial tcp 10.134.0.32:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5859e-predictor-76c758fc77-sq8sr |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.31:8643/healthz": dial tcp 10.134.0.31:8643: connect: connection refused | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-492f6 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-492f6": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-492f6-predictor-598d856bcd |
SuccessfulCreate |
Created pod: error-404-isvc-492f6-predictor-598d856bcd-kxgf8 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-35019-predictor-6647d995d5-4jxvp |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-35019-predictor-6647d995d5-4jxvp |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-492f6 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-492f6": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-492f6": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-35019-predictor-86b8b845bd-2l9hx |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-35019-predictor-86b8b845bd-2l9hx |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-35019-8565bdbbb7-fcg64 |
Killing |
Stopping container splitter-graph-35019 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-492f6-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-492f6-predictor-598d856bcd from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-492f6-predictor-6698d5fcc5 |
SuccessfulCreate |
Created pod: success-200-isvc-492f6-predictor-6698d5fcc5-m8sgw | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-492f6-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-492f6-predictor-6698d5fcc5 from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-492f6 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-492f6": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-492f6": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-492f6 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-492f6": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
multus |
error-404-isvc-492f6-predictor-598d856bcd-kxgf8 |
AddedInterface |
Add eth0 [10.134.0.38/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-492f6-predictor-6698d5fcc5-m8sgw |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-492f6-predictor-598d856bcd-kxgf8 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-492f6-predictor-598d856bcd-kxgf8 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-492f6-predictor-598d856bcd-kxgf8 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-492f6-predictor-6698d5fcc5-m8sgw |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-492f6-predictor-6698d5fcc5-m8sgw |
AddedInterface |
Add eth0 [10.134.0.37/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-492f6-predictor-6698d5fcc5-m8sgw |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-492f6-predictor-6698d5fcc5-m8sgw |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-492f6-predictor-6698d5fcc5-m8sgw |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1733" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-492f6-predictor-598d856bcd-kxgf8 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-492f6-predictor-598d856bcd-kxgf8 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-492f6-predictor-6698d5fcc5-m8sgw |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-492f6-predictor-598d856bcd-kxgf8 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1733" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-35019-predictor-86b8b845bd-2l9hx |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.34:8643/healthz": dial tcp 10.134.0.34:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-35019-predictor-6647d995d5-4jxvp |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.33:8643/healthz": dial tcp 10.134.0.33:8643: connect: connection refused | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-492f6-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-492f6-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-5859e-7c56564db7-krtf4 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-575ad-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-575ad-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-575ad-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-575ad-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-35019-8565bdbbb7-fcg64 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-575ad-predictor-7d77b767d4-xbql6 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.36:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-575ad-predictor-9f95b5ddf-p6vcw |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.35:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-575ad |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-575ad |
InferenceServiceReady |
InferenceService [error-404-isvc-575ad] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-492f6-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-492f6-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-492f6 |
InferenceServiceReady |
InferenceService [success-200-isvc-492f6] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-575ad |
InferenceServiceReady |
InferenceService [success-200-isvc-575ad] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-492f6 |
InferenceServiceReady |
InferenceService [error-404-isvc-492f6] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-575ad |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-492f6 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-492f6 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-575ad-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-575ad-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-575ad |
ScalingReplicaSet |
Scaled up replica set switch-graph-575ad-64d456d77c from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-492f6 |
ScalingReplicaSet |
Scaled up replica set splitter-graph-492f6-67c988957c from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-575ad |
InferenceGraphReady |
InferenceGraph [switch-graph-575ad] is Ready | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-492f6 |
InferenceGraphReady |
InferenceGraph [splitter-graph-492f6] is Ready | |
kserve-ci-e2e-test |
kubelet |
switch-graph-575ad-64d456d77c-rsdgn |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-575ad-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-575ad-64d456d77c |
SuccessfulCreate |
Created pod: switch-graph-575ad-64d456d77c-rsdgn | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-492f6-67c988957c |
SuccessfulCreate |
Created pod: splitter-graph-492f6-67c988957c-hcprv | |
kserve-ci-e2e-test |
kubelet |
switch-graph-575ad-64d456d77c-rsdgn |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:8345cd1e0b8ad942d68e2fe65c99d692fac6452f8ea219efb6700aaf29b3e2fa" already present on machine | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-492f6-67c988957c-hcprv |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:8345cd1e0b8ad942d68e2fe65c99d692fac6452f8ea219efb6700aaf29b3e2fa" already present on machine | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-492f6-67c988957c-hcprv |
Started |
Started container splitter-graph-492f6 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-575ad-64d456d77c-rsdgn |
Created |
Created container: switch-graph-575ad | |
kserve-ci-e2e-test |
multus |
switch-graph-575ad-64d456d77c-rsdgn |
AddedInterface |
Add eth0 [10.132.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
splitter-graph-492f6-67c988957c-hcprv |
AddedInterface |
Add eth0 [10.132.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-575ad-64d456d77c-rsdgn |
Started |
Started container switch-graph-575ad | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-492f6-67c988957c-hcprv |
Created |
Created container: splitter-graph-492f6 | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-575ad-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-575ad-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-575ad |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-575ad |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-492f6-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-492f6-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-575ad |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-492f6-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-575ad |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-492f6-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-492f6 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-492f6 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-492f6-predictor-598d856bcd-kxgf8 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-492f6-predictor-6698d5fcc5-m8sgw |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-492f6-predictor-6698d5fcc5-m8sgw |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-492f6-67c988957c-hcprv |
Killing |
Stopping container splitter-graph-492f6 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-492f6-predictor-598d856bcd-kxgf8 |
Killing |
Stopping container kube-rbac-proxy | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-492f6-predictor-598d856bcd-kxgf8 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.38:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-492f6-predictor-6698d5fcc5-m8sgw |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.37:8643/healthz": dial tcp 10.134.0.37:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-492f6-predictor-6698d5fcc5-m8sgw |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.37:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-492f6-predictor-598d856bcd-kxgf8 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.38:8643/healthz": dial tcp 10.134.0.38:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-492f6-67c988957c-hcprv |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-575ad-predictor-7d77b767d4-xbql6 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-575ad-predictor-7d77b767d4-xbql6 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
switch-graph-575ad-64d456d77c-rsdgn |
Killing |
Stopping container switch-graph-575ad | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-575ad-predictor-9f95b5ddf-p6vcw |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-575ad-predictor-9f95b5ddf-p6vcw |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-575ad-predictor-9f95b5ddf-p6vcw |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.35:8643/healthz": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) | |
| (x5) | kserve-ci-e2e-test |
kubelet |
switch-graph-575ad-64d456d77c-rsdgn |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |