| Time | Namespace | Component | RelatedObject | Reason | Message |
|---|---|---|---|---|---|
kserve-ci-e2e-test |
success-200-isvc-0a743-predictor-98f985b96-6lt86 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-0a743-predictor-98f985b96-6lt86 to ip-10-0-129-138.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-b12e8-predictor-57c478d4fd-lwqkw |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-b12e8-predictor-57c478d4fd-lwqkw to ip-10-0-129-138.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-0934b-74995788f-j8psd |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-0934b-74995788f-j8psd to ip-10-0-129-138.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-b12e8-predictor-665bb476f9-m2jk6 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6 to ip-10-0-143-36.ec2.internal | ||
kserve-ci-e2e-test |
isvc-xgboost-graph-predictor-555fc69d56-rqkpf |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-rqkpf to ip-10-0-137-188.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-f0182-predictor-54bbdb7d5b-vj5fx |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-f0182-predictor-54bbdb7d5b-vj5fx to ip-10-0-129-138.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-965a2-predictor-64fd5c48d4-rjkhz |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-965a2-predictor-64fd5c48d4-rjkhz to ip-10-0-129-138.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-afa2d-predictor-54c8dbf4d6-k86d2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-afa2d-predictor-54c8dbf4d6-k86d2 to ip-10-0-129-138.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-d2d23-predictor-77b7bd766d-vx77s |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-d2d23-predictor-77b7bd766d-vx77s to ip-10-0-129-138.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-afa2d-predictor-75594848c5-cc7sx |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx to ip-10-0-143-36.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-0a743-predictor-589d4b48df-5ss6c |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c to ip-10-0-143-36.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-d2d23-557d84f99-bwkd2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-d2d23-557d84f99-bwkd2 to ip-10-0-129-138.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-d2d23-predictor-658b5fdc7-kx72c |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c to ip-10-0-143-36.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-f1d40-predictor-58f666b764-89zbs |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-f1d40-predictor-58f666b764-89zbs to ip-10-0-129-138.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr to ip-10-0-143-36.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-f0182-predictor-6ff7979689-6jg5k |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k to ip-10-0-143-36.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-6242a-6df9fc859f-jpk45 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-6242a-6df9fc859f-jpk45 to ip-10-0-129-138.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-2-predictor-77b6fbc468-69w52 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-77b6fbc468-69w52 to ip-10-0-129-138.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-6242a-predictor-cbf7db67b-9cr5q |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-6242a-predictor-cbf7db67b-9cr5q to ip-10-0-129-138.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-f5066-predictor-569b8fdd8-tgl8h |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-f5066-predictor-569b8fdd8-tgl8h to ip-10-0-129-138.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj to ip-10-0-143-36.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-b12e8-db4cf88d6-fzbjk |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-b12e8-db4cf88d6-fzbjk to ip-10-0-129-138.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-f0182-68659b478c-prr6c |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-f0182-68659b478c-prr6c to ip-10-0-129-138.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-965a2-5ddc4d9f8b-lpck6 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-965a2-5ddc4d9f8b-lpck6 to ip-10-0-129-138.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-0934b-predictor-5949bbfc75-kftfs |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs to ip-10-0-143-36.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-f5066-predictor-645f7d7b85-fsdjq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq to ip-10-0-143-36.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-f1d40-78bdff7fd9-5sdnw |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-f1d40-78bdff7fd9-5sdnw to ip-10-0-129-138.ec2.internal | ||
kserve-ci-e2e-test |
model-chainer-79dd7c6579-pzmm2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/model-chainer-79dd7c6579-pzmm2 to ip-10-0-129-138.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-0a743-9d56587d4-sqlmx |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-0a743-9d56587d4-sqlmx to ip-10-0-129-138.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-0934b-predictor-5cb5448f5d-ftmm2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-0934b-predictor-5cb5448f5d-ftmm2 to ip-10-0-129-138.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-f5066-5dd8b587f8-6xmdh |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-f5066-5dd8b587f8-6xmdh to ip-10-0-129-138.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-afa2d-86d55575d-mkw79 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-afa2d-86d55575d-mkw79 to ip-10-0-129-138.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-1-predictor-65bbfc86dd-2qcj6 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-65bbfc86dd-2qcj6 to ip-10-0-137-188.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-965a2-predictor-6df948c77-gjp88 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88 to ip-10-0-143-36.ec2.internal | ||
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-1-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-1-predictor-65bbfc86dd from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-65bbfc86dd-2qcj6 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "isvc-sklearn-graph-1-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
isvc-xgboost-graph-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-xgboost-graph-predictor-555fc69d56 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-f0182-predictor-54bbdb7d5b |
SuccessfulCreate |
Created pod: success-200-isvc-f0182-predictor-54bbdb7d5b-vj5fx | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-f0182-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-f0182-predictor-54bbdb7d5b from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-f0182-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-f0182-predictor-6ff7979689 from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
UpdateFailed |
Failed to update status for InferenceService "isvc-sklearn-graph-1": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "isvc-sklearn-graph-1": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "isvc-sklearn-graph-1": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-1-predictor-65bbfc86dd |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-1-predictor-65bbfc86dd-2qcj6 | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-xgboost-graph-predictor-555fc69d56 |
SuccessfulCreate |
Created pod: isvc-xgboost-graph-predictor-555fc69d56-rqkpf | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-2-predictor-77b6fbc468 |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-2-predictor-77b6fbc468-69w52 | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-2-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-2-predictor-77b6fbc468 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-f0182-predictor-6ff7979689 |
SuccessfulCreate |
Created pod: error-404-isvc-f0182-predictor-6ff7979689-6jg5k | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-1-predictor-65bbfc86dd-2qcj6 |
AddedInterface |
Add eth0 [10.133.0.23/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0182-predictor-6ff7979689-6jg5k |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-f0182-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-65bbfc86dd-2qcj6 |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-65bbfc86dd-2qcj6 |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-77b6fbc468-69w52 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "isvc-sklearn-graph-2-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-65bbfc86dd-2qcj6 |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:13da3ee375cce7c9d4590e64ebb24ecf2e97ed4b2b64f9a4f2e3074ed4aafac1" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-f0182-predictor-54bbdb7d5b-vj5fx |
AddedInterface |
Add eth0 [10.134.0.16/23] from ovn-kubernetes | |
| (x2) | kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-rqkpf |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "isvc-xgboost-graph-predictor-serving-cert" not found |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f0182-predictor-54bbdb7d5b-vj5fx |
Pulling |
Pulling image "quay.io/opendatahub/success-200-isvc:odh-pr-1828" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-rqkpf |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-2-predictor-77b6fbc468-69w52 |
AddedInterface |
Add eth0 [10.134.0.17/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
isvc-xgboost-graph-predictor-555fc69d56-rqkpf |
AddedInterface |
Add eth0 [10.133.0.24/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0182-predictor-6ff7979689-6jg5k |
Pulling |
Pulling image "quay.io/opendatahub/error-404-isvc:odh-pr-1828" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-rqkpf |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-rqkpf |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:13da3ee375cce7c9d4590e64ebb24ecf2e97ed4b2b64f9a4f2e3074ed4aafac1" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-f0182-predictor-6ff7979689-6jg5k |
AddedInterface |
Add eth0 [10.132.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-77b6fbc468-69w52 |
Pulling |
Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:13da3ee375cce7c9d4590e64ebb24ecf2e97ed4b2b64f9a4f2e3074ed4aafac1" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-65bbfc86dd-2qcj6 |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1828" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-rqkpf |
Pulling |
Pulling image "kserve/xgbserver:latest" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0182-predictor-6ff7979689-6jg5k |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0182-predictor-6ff7979689-6jg5k |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0182-predictor-6ff7979689-6jg5k |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0182-predictor-6ff7979689-6jg5k |
Pulled |
Successfully pulled image "quay.io/opendatahub/error-404-isvc:odh-pr-1828" in 11.573s (11.573s including waiting). Image size: 1333943915 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-77b6fbc468-69w52 |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:13da3ee375cce7c9d4590e64ebb24ecf2e97ed4b2b64f9a4f2e3074ed4aafac1" in 12.685s (12.685s including waiting). Image size: 300978649 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f0182-predictor-54bbdb7d5b-vj5fx |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f0182-predictor-54bbdb7d5b-vj5fx |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-77b6fbc468-69w52 |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-77b6fbc468-69w52 |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f0182-predictor-54bbdb7d5b-vj5fx |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f0182-predictor-54bbdb7d5b-vj5fx |
Pulled |
Successfully pulled image "quay.io/opendatahub/success-200-isvc:odh-pr-1828" in 13.742s (13.742s including waiting). Image size: 1336828526 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0182-predictor-6ff7979689-6jg5k |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0182-predictor-6ff7979689-6jg5k |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.577s (2.577s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0182-predictor-6ff7979689-6jg5k |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f0182-predictor-54bbdb7d5b-vj5fx |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.251s (2.251s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f0182-predictor-54bbdb7d5b-vj5fx |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f0182-predictor-54bbdb7d5b-vj5fx |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-77b6fbc468-69w52 |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1828" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-77b6fbc468-69w52 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-77b6fbc468-69w52 |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1828" in 6.29s (6.29s including waiting). Image size: 1555733430 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-77b6fbc468-69w52 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-77b6fbc468-69w52 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-77b6fbc468-69w52 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-77b6fbc468-69w52 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-65bbfc86dd-2qcj6 |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1828" in 24.127s (24.127s including waiting). Image size: 1555733430 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-65bbfc86dd-2qcj6 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-65bbfc86dd-2qcj6 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-65bbfc86dd-2qcj6 |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-rqkpf |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-rqkpf |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-rqkpf |
Pulled |
Successfully pulled image "kserve/xgbserver:latest" in 23.456s (23.456s including waiting). Image size: 1307355899 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-rqkpf |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-65bbfc86dd-2qcj6 |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 3.089s (3.089s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-rqkpf |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-65bbfc86dd-2qcj6 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-65bbfc86dd-2qcj6 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-rqkpf |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-rqkpf |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.868s (2.868s including waiting). Image size: 211946088 bytes. | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f0182-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f0182-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f0182-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f0182-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-f0182-predictor-54bbdb7d5b-vj5fx |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.16:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0182-predictor-6ff7979689-6jg5k |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.35:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-f0182 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f0182 |
InferenceServiceReady |
InferenceService [error-404-isvc-f0182] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-f0182 |
InferenceServiceReady |
InferenceService [success-200-isvc-f0182] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f0182 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f0182-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f0182-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-f0182-68659b478c |
SuccessfulCreate |
Created pod: switch-graph-f0182-68659b478c-prr6c | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-f0182 |
ScalingReplicaSet |
Scaled up replica set switch-graph-f0182-68659b478c from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-f0182 |
InferenceGraphReady |
InferenceGraph [switch-graph-f0182] is Ready | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-f0182 |
UpdateFailed |
Failed to update status for InferenceGraph "switch-graph-f0182": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "switch-graph-f0182": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-f0182 |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "switch-graph-f0182": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
switch-graph-f0182-68659b478c-prr6c |
Pulling |
Pulling image "quay.io/opendatahub/kserve-router@sha256:41cb8bf9ff6c7df05622085595fdb4c1e1d3a571d74d4bdea37a3b72ad46799b" | |
kserve-ci-e2e-test |
multus |
switch-graph-f0182-68659b478c-prr6c |
AddedInterface |
Add eth0 [10.134.0.18/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-f0182-68659b478c-prr6c |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-router@sha256:41cb8bf9ff6c7df05622085595fdb4c1e1d3a571d74d4bdea37a3b72ad46799b" in 2.312s (2.312s including waiting). Image size: 218226816 bytes. | |
kserve-ci-e2e-test |
kubelet |
switch-graph-f0182-68659b478c-prr6c |
Started |
Started container switch-graph-f0182 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-f0182-68659b478c-prr6c |
Created |
Created container: switch-graph-f0182 | |
| (x8) | kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-rqkpf |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.24:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-0a743 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-0a743": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-0a743 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-0a743": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-0a743": the object has been modified; please apply your changes to the latest version and try again | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f0182-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
switch-graph-f0182-68659b478c-prr6c |
Killing |
Stopping container switch-graph-f0182 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0182-predictor-6ff7979689-6jg5k |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-0a743-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-0a743-predictor-98f985b96 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-0a743-predictor-98f985b96 |
SuccessfulCreate |
Created pod: success-200-isvc-0a743-predictor-98f985b96-6lt86 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f0182-predictor-54bbdb7d5b-vj5fx |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0182-predictor-6ff7979689-6jg5k |
Killing |
Stopping container kserve-container | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f0182-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f0182-predictor-54bbdb7d5b-vj5fx |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0a743-predictor-98f985b96-6lt86 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0a743-predictor-98f985b96-6lt86 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-0a743 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-0a743": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-0a743 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-0a743": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-0a743": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-0a743-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-0a743-predictor-589d4b48df from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-0a743-predictor-589d4b48df |
SuccessfulCreate |
Created pod: error-404-isvc-0a743-predictor-589d4b48df-5ss6c | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0a743-predictor-589d4b48df-5ss6c |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-0a743-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
multus |
success-200-isvc-0a743-predictor-98f985b96-6lt86 |
AddedInterface |
Add eth0 [10.134.0.19/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0a743-predictor-98f985b96-6lt86 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1828" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0a743-predictor-98f985b96-6lt86 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0a743-predictor-98f985b96-6lt86 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0a743-predictor-98f985b96-6lt86 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0a743-predictor-589d4b48df-5ss6c |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-0a743-predictor-589d4b48df-5ss6c |
AddedInterface |
Add eth0 [10.132.0.36/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0a743-predictor-589d4b48df-5ss6c |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0a743-predictor-589d4b48df-5ss6c |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1828" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0a743-predictor-589d4b48df-5ss6c |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0a743-predictor-589d4b48df-5ss6c |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0a743-predictor-589d4b48df-5ss6c |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0182-predictor-6ff7979689-6jg5k |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.35:8643/healthz": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0182-predictor-6ff7979689-6jg5k |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.35:8080: i/o timeout | |
| (x10) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-77b6fbc468-69w52 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.17:8080: connect: connection refused |
| (x10) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
InferenceServiceReady |
InferenceService [isvc-xgboost-graph] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-2] is Ready | |
| (x10) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-1] is Ready | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-f0182-68659b478c-prr6c |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-0a743-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-0a743-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-0a743-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-0a743-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-0a743 |
InferenceServiceReady |
InferenceService [success-200-isvc-0a743] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-0a743 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-0a743 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-0a743 |
InferenceServiceReady |
InferenceService [error-404-isvc-0a743] is Ready | |
kserve-ci-e2e-test |
InferenceGraphController |
model-chainer |
InferenceGraphReady |
InferenceGraph [model-chainer] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
model-chainer |
ScalingReplicaSet |
Scaled up replica set model-chainer-79dd7c6579 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
model-chainer-79dd7c6579 |
SuccessfulCreate |
Created pod: model-chainer-79dd7c6579-pzmm2 | |
kserve-ci-e2e-test |
kubelet |
model-chainer-79dd7c6579-pzmm2 |
Started |
Started container model-chainer | |
kserve-ci-e2e-test |
kubelet |
model-chainer-79dd7c6579-pzmm2 |
Created |
Created container: model-chainer | |
kserve-ci-e2e-test |
kubelet |
model-chainer-79dd7c6579-pzmm2 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:41cb8bf9ff6c7df05622085595fdb4c1e1d3a571d74d4bdea37a3b72ad46799b" already present on machine | |
kserve-ci-e2e-test |
multus |
model-chainer-79dd7c6579-pzmm2 |
AddedInterface |
Add eth0 [10.134.0.20/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-65bbfc86dd-2qcj6 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
model-chainer-79dd7c6579-pzmm2 |
Killing |
Stopping container model-chainer | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-f1d40-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-f1d40-predictor-58f666b764 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-f1d40-predictor-58f666b764 |
SuccessfulCreate |
Created pod: success-200-isvc-f1d40-predictor-58f666b764-89zbs | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-rqkpf |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetScale |
deployments.apps "isvc-sklearn-graph-1-predictor" not found | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedUpdateStatus |
horizontalpodautoscalers.autoscaling "isvc-sklearn-graph-1-predictor" not found | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-rqkpf |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-65bbfc86dd-2qcj6 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f1d40 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-f1d40": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f1d40 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-f1d40": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-f1d40": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-77b6fbc468-69w52 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-77b6fbc468-69w52 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-f1d40-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-f1d40-predictor-749bd5f9df from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-f1d40-predictor-749bd5f9df |
SuccessfulCreate |
Created pod: error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
switch-graph-0a743-9d56587d4-sqlmx |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-0a743-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj |
Started |
Started container kube-rbac-proxy | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-0a743-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-65bbfc86dd-2qcj6 |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.23:8643/healthz": dial tcp 10.133.0.23:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-0a743 |
InferenceGraphReady |
InferenceGraph [switch-graph-0a743] is Ready | |
| (x2) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-f1d40-predictor-58f666b764-89zbs |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-f1d40-predictor-serving-cert" not found |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-0a743-9d56587d4 |
SuccessfulCreate |
Created pod: switch-graph-0a743-9d56587d4-sqlmx | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj |
Started |
Started container kserve-container | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-0a743-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1828" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj |
AddedInterface |
Add eth0 [10.132.0.37/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-0a743 |
ScalingReplicaSet |
Scaled up replica set switch-graph-0a743-9d56587d4 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f1d40-predictor-58f666b764-89zbs |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-f1d40-predictor-58f666b764-89zbs |
AddedInterface |
Add eth0 [10.134.0.21/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f1d40-predictor-58f666b764-89zbs |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
switch-graph-0a743-9d56587d4-sqlmx |
Started |
Started container switch-graph-0a743 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-0a743-9d56587d4-sqlmx |
Created |
Created container: switch-graph-0a743 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-0a743-9d56587d4-sqlmx |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:41cb8bf9ff6c7df05622085595fdb4c1e1d3a571d74d4bdea37a3b72ad46799b" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f1d40-predictor-58f666b764-89zbs |
Created |
Created container: kube-rbac-proxy | |
| (x11) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-65bbfc86dd-2qcj6 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.23:8080: connect: connection refused |
kserve-ci-e2e-test |
multus |
switch-graph-0a743-9d56587d4-sqlmx |
AddedInterface |
Add eth0 [10.134.0.22/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f1d40-predictor-58f666b764-89zbs |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1828" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f1d40-predictor-58f666b764-89zbs |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f1d40-predictor-58f666b764-89zbs |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-77b6fbc468-69w52 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.17:8643/healthz": dial tcp 10.134.0.17:8643: connect: connection refused | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-0a743-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-0a743-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
model-chainer-79dd7c6579-pzmm2 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f1d40-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f1d40-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f1d40-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f1d40-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-0a743 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-0a743 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-f1d40-predictor-58f666b764-89zbs |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.21:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f1d40 |
InferenceServiceReady |
InferenceService [error-404-isvc-f1d40] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-f1d40 |
InferenceServiceReady |
InferenceService [success-200-isvc-f1d40] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f1d40 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-f1d40 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-f1d40 |
InferenceGraphReady |
InferenceGraph [sequence-graph-f1d40] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-f1d40 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-f1d40-78bdff7fd9 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-f1d40-78bdff7fd9 |
SuccessfulCreate |
Created pod: sequence-graph-f1d40-78bdff7fd9-5sdnw | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f1d40-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f1d40-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
sequence-graph-f1d40-78bdff7fd9-5sdnw |
Started |
Started container sequence-graph-f1d40 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-f1d40-78bdff7fd9-5sdnw |
Created |
Created container: sequence-graph-f1d40 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-f1d40-78bdff7fd9-5sdnw |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:41cb8bf9ff6c7df05622085595fdb4c1e1d3a571d74d4bdea37a3b72ad46799b" already present on machine | |
kserve-ci-e2e-test |
multus |
sequence-graph-f1d40-78bdff7fd9-5sdnw |
AddedInterface |
Add eth0 [10.134.0.23/23] from ovn-kubernetes | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f1d40-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f1d40-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-f1d40 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-f1d40 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
switch-graph-0a743-9d56587d4-sqlmx |
Killing |
Stopping container switch-graph-0a743 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0a743-predictor-98f985b96-6lt86 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0a743-predictor-98f985b96-6lt86 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-b12e8 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-b12e8": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-b12e8-predictor-57c478d4fd |
SuccessfulCreate |
Created pod: success-200-isvc-b12e8-predictor-57c478d4fd-lwqkw | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-b12e8-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-b12e8-predictor-57c478d4fd from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-b12e8 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-b12e8": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-b12e8": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-b12e8-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-b12e8-predictor-665bb476f9 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-b12e8-predictor-665bb476f9 |
SuccessfulCreate |
Created pod: error-404-isvc-b12e8-predictor-665bb476f9-m2jk6 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b12e8-predictor-57c478d4fd-lwqkw |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-b12e8-predictor-57c478d4fd-lwqkw |
AddedInterface |
Add eth0 [10.134.0.24/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
error-404-isvc-b12e8-predictor-665bb476f9-m2jk6 |
AddedInterface |
Add eth0 [10.132.0.38/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b12e8-predictor-665bb476f9-m2jk6 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1828" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b12e8-predictor-57c478d4fd-lwqkw |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0a743-predictor-589d4b48df-5ss6c |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b12e8-predictor-665bb476f9-m2jk6 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b12e8-predictor-665bb476f9-m2jk6 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b12e8-predictor-665bb476f9-m2jk6 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b12e8-predictor-665bb476f9-m2jk6 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b12e8-predictor-665bb476f9-m2jk6 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b12e8-predictor-57c478d4fd-lwqkw |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b12e8-predictor-57c478d4fd-lwqkw |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0a743-predictor-589d4b48df-5ss6c |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b12e8-predictor-57c478d4fd-lwqkw |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1828" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b12e8-predictor-57c478d4fd-lwqkw |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0a743-predictor-589d4b48df-5ss6c |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.36:8643/healthz": dial tcp 10.132.0.36:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0a743-predictor-98f985b96-6lt86 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.19:8643/healthz": dial tcp 10.134.0.19:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-0a743-predictor-589d4b48df-5ss6c |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.36:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-0a743-predictor-98f985b96-6lt86 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.19:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-0a743-9d56587d4-sqlmx |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-b12e8-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-b12e8-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-b12e8-predictor-57c478d4fd-lwqkw |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.24:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-b12e8-predictor-665bb476f9-m2jk6 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.38:8080: connect: connection refused |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-b12e8 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-b12e8 |
InferenceServiceReady |
InferenceService [success-200-isvc-b12e8] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-b12e8 |
InferenceServiceReady |
InferenceService [error-404-isvc-b12e8] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-b12e8 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-6242a |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-6242a": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-6242a": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f1d40-predictor-58f666b764-89zbs |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6242a-predictor-cbf7db67b-9cr5q |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-6242a-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-f1d40-78bdff7fd9-5sdnw |
Killing |
Stopping container sequence-graph-f1d40 | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-b12e8-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj |
Killing |
Stopping container kube-rbac-proxy | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-b12e8-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-6242a |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-6242a": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-6242a |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-6242a": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-6242a": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-6242a-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-6242a-predictor-6d94cbd9cc from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-6242a-predictor-6d94cbd9cc |
SuccessfulCreate |
Created pod: error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-6242a-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-6242a-predictor-cbf7db67b from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-6242a |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-6242a": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-6242a-predictor-cbf7db67b |
SuccessfulCreate |
Created pod: success-200-isvc-6242a-predictor-cbf7db67b-9cr5q | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f1d40-predictor-58f666b764-89zbs |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6242a-predictor-cbf7db67b-9cr5q |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6242a-predictor-cbf7db67b-9cr5q |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr |
Created |
Created container: kube-rbac-proxy | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-b12e8-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-b12e8-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-b12e8-db4cf88d6 |
SuccessfulCreate |
Created pod: ensemble-graph-b12e8-db4cf88d6-fzbjk | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-b12e8 |
InferenceGraphReady |
InferenceGraph [ensemble-graph-b12e8] is Ready | |
kserve-ci-e2e-test |
multus |
error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr |
AddedInterface |
Add eth0 [10.132.0.39/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-b12e8 |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-b12e8-db4cf88d6 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
success-200-isvc-6242a-predictor-cbf7db67b-9cr5q |
AddedInterface |
Add eth0 [10.134.0.25/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-b12e8-db4cf88d6-fzbjk |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "ensemble-graph-b12e8-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6242a-predictor-cbf7db67b-9cr5q |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1828" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6242a-predictor-cbf7db67b-9cr5q |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1828" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6242a-predictor-cbf7db67b-9cr5q |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6242a-predictor-cbf7db67b-9cr5q |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
ensemble-graph-b12e8-db4cf88d6-fzbjk |
AddedInterface |
Add eth0 [10.134.0.26/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-b12e8-db4cf88d6-fzbjk |
Created |
Created container: ensemble-graph-b12e8 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-b12e8-db4cf88d6-fzbjk |
Started |
Started container ensemble-graph-b12e8 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-b12e8-db4cf88d6-fzbjk |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:41cb8bf9ff6c7df05622085595fdb4c1e1d3a571d74d4bdea37a3b72ad46799b" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.37:8643/healthz": dial tcp 10.132.0.37:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.37:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d2d23-predictor-658b5fdc7-kx72c |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-d2d23-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-d2d23-predictor-77b7bd766d from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d2d23 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-d2d23": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d2d23 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-d2d23": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d2d23 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-d2d23": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-d2d23": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d2d23 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-d2d23": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-d2d23": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-d2d23-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-d2d23-predictor-658b5fdc7 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-d2d23-predictor-658b5fdc7 |
SuccessfulCreate |
Created pod: error-404-isvc-d2d23-predictor-658b5fdc7-kx72c | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d2d23-predictor-658b5fdc7-kx72c |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d2d23-predictor-658b5fdc7-kx72c |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d2d23-predictor-658b5fdc7-kx72c |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d2d23-predictor-658b5fdc7-kx72c |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b12e8-predictor-57c478d4fd-lwqkw |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d2d23-predictor-658b5fdc7-kx72c |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1828" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-d2d23-predictor-658b5fdc7-kx72c |
AddedInterface |
Add eth0 [10.132.0.40/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b12e8-predictor-57c478d4fd-lwqkw |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b12e8-predictor-665bb476f9-m2jk6 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b12e8-predictor-665bb476f9-m2jk6 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-b12e8-db4cf88d6-fzbjk |
Killing |
Stopping container ensemble-graph-b12e8 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-d2d23-predictor-77b7bd766d |
SuccessfulCreate |
Created pod: success-200-isvc-d2d23-predictor-77b7bd766d-vx77s | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d2d23-predictor-77b7bd766d-vx77s |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-d2d23-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d2d23-predictor-77b7bd766d-vx77s |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d2d23-predictor-77b7bd766d-vx77s |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d2d23-predictor-77b7bd766d-vx77s |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d2d23-predictor-77b7bd766d-vx77s |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d2d23-predictor-77b7bd766d-vx77s |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1828" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-d2d23-predictor-77b7bd766d-vx77s |
AddedInterface |
Add eth0 [10.134.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d2d23-predictor-77b7bd766d-vx77s |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b12e8-predictor-57c478d4fd-lwqkw |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.24:8643/healthz": dial tcp 10.134.0.24:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b12e8-predictor-665bb476f9-m2jk6 |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.38:8643/healthz": dial tcp 10.132.0.38:8643: connect: connection refused | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d2d23-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d2d23-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-f1d40-78bdff7fd9-5sdnw |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-6242a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-6242a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-6242a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-6242a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-6242a-predictor-cbf7db67b-9cr5q |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.25:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-b12e8-db4cf88d6-fzbjk |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.39:8080: connect: connection refused |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d2d23-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d2d23-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-6242a |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-6242a |
InferenceServiceReady |
InferenceService [error-404-isvc-6242a] is Ready | |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-d2d23-predictor-658b5fdc7-kx72c |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.40:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-6242a |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-6242a |
InferenceServiceReady |
InferenceService [success-200-isvc-6242a] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d2d23 |
InferenceServiceReady |
InferenceService [error-404-isvc-d2d23] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d2d23 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d2d23 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d2d23 |
InferenceServiceReady |
InferenceService [success-200-isvc-d2d23] is Ready | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-6242a |
InferenceGraphReady |
InferenceGraph [sequence-graph-6242a] is Ready | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-6242a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-6242a |
ScalingReplicaSet |
Scaled up replica set sequence-graph-6242a-6df9fc859f from 0 to 1 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-6242a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-6242a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-6242a-6df9fc859f |
SuccessfulCreate |
Created pod: sequence-graph-6242a-6df9fc859f-jpk45 | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-6242a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
sequence-graph-6242a-6df9fc859f-jpk45 |
Created |
Created container: sequence-graph-6242a | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-6242a-6df9fc859f-jpk45 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "sequence-graph-6242a-serving-cert" not found | |
kserve-ci-e2e-test |
multus |
sequence-graph-6242a-6df9fc859f-jpk45 |
AddedInterface |
Add eth0 [10.134.0.28/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-6242a-6df9fc859f-jpk45 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:41cb8bf9ff6c7df05622085595fdb4c1e1d3a571d74d4bdea37a3b72ad46799b" already present on machine | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-6242a-6df9fc859f-jpk45 |
Started |
Started container sequence-graph-6242a | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6242a-predictor-cbf7db67b-9cr5q |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-6242a-6df9fc859f-jpk45 |
Killing |
Stopping container sequence-graph-6242a | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6242a-predictor-cbf7db67b-9cr5q |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-0934b-predictor-5cb5448f5d |
SuccessfulCreate |
Created pod: success-200-isvc-0934b-predictor-5cb5448f5d-ftmm2 | |
kserve-ci-e2e-test |
multus |
error-404-isvc-0934b-predictor-5949bbfc75-kftfs |
AddedInterface |
Add eth0 [10.132.0.41/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-0934b-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-0934b-predictor-5cb5448f5d from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-d2d23-557d84f99-bwkd2 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "ensemble-graph-d2d23-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-0934b-predictor-5949bbfc75 |
SuccessfulCreate |
Created pod: error-404-isvc-0934b-predictor-5949bbfc75-kftfs | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-0934b-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-0934b-predictor-5949bbfc75 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0934b-predictor-5949bbfc75-kftfs |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0934b-predictor-5949bbfc75-kftfs |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-0934b |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-0934b": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-0934b": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0934b-predictor-5949bbfc75-kftfs |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0934b-predictor-5949bbfc75-kftfs |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1828" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0934b-predictor-5cb5448f5d-ftmm2 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-0934b-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-0934b |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-0934b": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0934b-predictor-5949bbfc75-kftfs |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0934b-predictor-5949bbfc75-kftfs |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-d2d23 |
InferenceGraphReady |
InferenceGraph [ensemble-graph-d2d23] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-d2d23 |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-d2d23-557d84f99 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-d2d23-557d84f99 |
SuccessfulCreate |
Created pod: ensemble-graph-d2d23-557d84f99-bwkd2 | |
kserve-ci-e2e-test |
multus |
success-200-isvc-0934b-predictor-5cb5448f5d-ftmm2 |
AddedInterface |
Add eth0 [10.134.0.29/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
ensemble-graph-d2d23-557d84f99-bwkd2 |
AddedInterface |
Add eth0 [10.134.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-d2d23-557d84f99-bwkd2 |
Started |
Started container ensemble-graph-d2d23 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-d2d23-557d84f99-bwkd2 |
Created |
Created container: ensemble-graph-d2d23 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0934b-predictor-5cb5448f5d-ftmm2 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1828" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0934b-predictor-5cb5448f5d-ftmm2 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0934b-predictor-5cb5448f5d-ftmm2 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0934b-predictor-5cb5448f5d-ftmm2 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0934b-predictor-5cb5448f5d-ftmm2 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0934b-predictor-5cb5448f5d-ftmm2 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-d2d23-557d84f99-bwkd2 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:41cb8bf9ff6c7df05622085595fdb4c1e1d3a571d74d4bdea37a3b72ad46799b" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6242a-predictor-cbf7db67b-9cr5q |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.25:8643/healthz": dial tcp 10.134.0.25:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.39:8643/healthz": dial tcp 10.132.0.39:8643: connect: connection refused | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d2d23-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d2d23-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d2d23-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d2d23-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-6242a-6df9fc859f-jpk45 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-d2d23 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-d2d23 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-0934b-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-0934b-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-0934b-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-0934b-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-0934b |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-0934b |
InferenceServiceReady |
InferenceService [error-404-isvc-0934b] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-0934b |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-0934b |
InferenceServiceReady |
InferenceService [success-200-isvc-0934b] is Ready | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-0934b |
InferenceGraphReady |
InferenceGraph [sequence-graph-0934b] is Ready | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-0934b-74995788f-j8psd |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:41cb8bf9ff6c7df05622085595fdb4c1e1d3a571d74d4bdea37a3b72ad46799b" already present on machine | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-0934b-74995788f-j8psd |
Created |
Created container: sequence-graph-0934b | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-0934b-74995788f |
SuccessfulCreate |
Created pod: sequence-graph-0934b-74995788f-j8psd | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-0934b |
ScalingReplicaSet |
Scaled up replica set sequence-graph-0934b-74995788f from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-0934b-74995788f-j8psd |
Started |
Started container sequence-graph-0934b | |
kserve-ci-e2e-test |
multus |
sequence-graph-0934b-74995788f-j8psd |
AddedInterface |
Add eth0 [10.134.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-0934b |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-0934b |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-0934b-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-0934b-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-0934b |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-0934b-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-0934b-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-0934b |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-d2d23-557d84f99-bwkd2 |
Killing |
Stopping container ensemble-graph-d2d23 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f5066-predictor-569b8fdd8-tgl8h |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d2d23-predictor-77b7bd766d-vx77s |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f5066-predictor-645f7d7b85-fsdjq |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1828" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f5066-predictor-645f7d7b85-fsdjq |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f5066-predictor-645f7d7b85-fsdjq |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f5066-predictor-645f7d7b85-fsdjq |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f5066-predictor-645f7d7b85-fsdjq |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f5066-predictor-645f7d7b85-fsdjq |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d2d23-predictor-658b5fdc7-kx72c |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d2d23-predictor-658b5fdc7-kx72c |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d2d23-predictor-77b7bd766d-vx77s |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-f5066-predictor-645f7d7b85 |
SuccessfulCreate |
Created pod: error-404-isvc-f5066-predictor-645f7d7b85-fsdjq | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-f5066-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-f5066-predictor-645f7d7b85 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
success-200-isvc-f5066-predictor-569b8fdd8-tgl8h |
AddedInterface |
Add eth0 [10.134.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f5066-predictor-569b8fdd8-tgl8h |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1828" already present on machine | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-f5066-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-f5066-predictor-569b8fdd8 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-f5066-predictor-569b8fdd8 |
SuccessfulCreate |
Created pod: success-200-isvc-f5066-predictor-569b8fdd8-tgl8h | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f5066-predictor-569b8fdd8-tgl8h |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-f5066-predictor-645f7d7b85-fsdjq |
AddedInterface |
Add eth0 [10.132.0.42/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f5066-predictor-569b8fdd8-tgl8h |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f5066-predictor-569b8fdd8-tgl8h |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f5066-predictor-569b8fdd8-tgl8h |
Started |
Started container kserve-container | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-d2d23-predictor-77b7bd766d-vx77s |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.27:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d2d23-predictor-77b7bd766d-vx77s |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.27:8643/healthz": dial tcp 10.134.0.27:8643: connect: connection refused | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f5066-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f5066-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-d2d23-557d84f99-bwkd2 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f5066-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f5066-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-f5066-predictor-569b8fdd8-tgl8h |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.32:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-f5066-predictor-645f7d7b85-fsdjq |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.42:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-f5066 |
InferenceServiceReady |
InferenceService [success-200-isvc-f5066] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-f5066 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f5066 |
InferenceServiceReady |
InferenceService [error-404-isvc-f5066] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f5066 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
kubelet |
sequence-graph-0934b-74995788f-j8psd |
Killing |
Stopping container sequence-graph-0934b | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-965a2-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-965a2-predictor-64fd5c48d4 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0934b-predictor-5cb5448f5d-ftmm2 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0934b-predictor-5cb5448f5d-ftmm2 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0934b-predictor-5949bbfc75-kftfs |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-965a2-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-965a2-predictor-6df948c77 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0934b-predictor-5949bbfc75-kftfs |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-965a2-predictor-64fd5c48d4 |
SuccessfulCreate |
Created pod: success-200-isvc-965a2-predictor-64fd5c48d4-rjkhz | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f5066-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-f5066 |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "splitter-graph-f5066": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-965a2-predictor-64fd5c48d4-rjkhz |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-965a2-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-965a2-predictor-6df948c77-gjp88 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-965a2-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-965a2-predictor-6df948c77 |
SuccessfulCreate |
Created pod: error-404-isvc-965a2-predictor-6df948c77-gjp88 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-965a2 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-965a2": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-965a2": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
multus |
success-200-isvc-965a2-predictor-64fd5c48d4-rjkhz |
AddedInterface |
Add eth0 [10.134.0.33/23] from ovn-kubernetes | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f5066-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-965a2 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-965a2": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
multus |
splitter-graph-f5066-5dd8b587f8-6xmdh |
AddedInterface |
Add eth0 [10.134.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-f5066-5dd8b587f8-6xmdh |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:41cb8bf9ff6c7df05622085595fdb4c1e1d3a571d74d4bdea37a3b72ad46799b" already present on machine | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-f5066 |
UpdateFailed |
Failed to update status for InferenceGraph "splitter-graph-f5066": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "splitter-graph-f5066": the object has been modified; please apply your changes to the latest version and try again | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f5066-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f5066-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-f5066 |
InferenceGraphReady |
InferenceGraph [splitter-graph-f5066] is Ready | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-f5066-5dd8b587f8-6xmdh |
Created |
Created container: splitter-graph-f5066 | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-f5066-5dd8b587f8-6xmdh |
Started |
Started container splitter-graph-f5066 | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-f5066 |
ScalingReplicaSet |
Scaled up replica set splitter-graph-f5066-5dd8b587f8 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-f5066-5dd8b587f8 |
SuccessfulCreate |
Created pod: splitter-graph-f5066-5dd8b587f8-6xmdh | |
kserve-ci-e2e-test |
multus |
error-404-isvc-965a2-predictor-6df948c77-gjp88 |
AddedInterface |
Add eth0 [10.132.0.43/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-965a2-predictor-64fd5c48d4-rjkhz |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-965a2-predictor-64fd5c48d4-rjkhz |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-965a2-predictor-64fd5c48d4-rjkhz |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-965a2-predictor-64fd5c48d4-rjkhz |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-965a2-predictor-64fd5c48d4-rjkhz |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-965a2-predictor-6df948c77-gjp88 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1828" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-965a2-predictor-6df948c77-gjp88 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-965a2-predictor-6df948c77-gjp88 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-965a2-predictor-6df948c77-gjp88 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-965a2-predictor-6df948c77-gjp88 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-965a2-predictor-6df948c77-gjp88 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-965a2-predictor-64fd5c48d4-rjkhz |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1828" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0934b-predictor-5949bbfc75-kftfs |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.41:8643/healthz": dial tcp 10.132.0.41:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0934b-predictor-5cb5448f5d-ftmm2 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.29:8643/healthz": dial tcp 10.134.0.29:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-0934b-predictor-5949bbfc75-kftfs |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.41:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-0934b-predictor-5cb5448f5d-ftmm2 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.29:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f5066-predictor-645f7d7b85-fsdjq |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-afa2d |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-afa2d": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-afa2d": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-afa2d-predictor-54c8dbf4d6 |
SuccessfulCreate |
Created pod: success-200-isvc-afa2d-predictor-54c8dbf4d6-k86d2 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f5066-predictor-569b8fdd8-tgl8h |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-f5066-5dd8b587f8-6xmdh |
Killing |
Stopping container splitter-graph-f5066 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f5066-predictor-569b8fdd8-tgl8h |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-afa2d |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-afa2d": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-afa2d": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f5066-predictor-645f7d7b85-fsdjq |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-afa2d |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-afa2d": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-afa2d-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-afa2d-predictor-54c8dbf4d6 from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-afa2d-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-afa2d-predictor-75594848c5 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
error-404-isvc-afa2d-predictor-75594848c5-cc7sx |
AddedInterface |
Add eth0 [10.132.0.44/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-afa2d-predictor-75594848c5-cc7sx |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1828" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-afa2d-predictor-75594848c5-cc7sx |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-afa2d-predictor-75594848c5-cc7sx |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-afa2d-predictor-75594848c5-cc7sx |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-afa2d-predictor-75594848c5 |
SuccessfulCreate |
Created pod: error-404-isvc-afa2d-predictor-75594848c5-cc7sx | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-afa2d-predictor-54c8dbf4d6-k86d2 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-afa2d-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-afa2d |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-afa2d": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-afa2d-predictor-54c8dbf4d6-k86d2 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-afa2d-predictor-54c8dbf4d6-k86d2 |
AddedInterface |
Add eth0 [10.134.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-afa2d-predictor-75594848c5-cc7sx |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-afa2d-predictor-54c8dbf4d6-k86d2 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-afa2d-predictor-54c8dbf4d6-k86d2 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1828" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-afa2d-predictor-54c8dbf4d6-k86d2 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-afa2d-predictor-54c8dbf4d6-k86d2 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-afa2d-predictor-54c8dbf4d6-k86d2 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-afa2d-predictor-75594848c5-cc7sx |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f5066-predictor-569b8fdd8-tgl8h |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.32:8643/healthz": dial tcp 10.134.0.32:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f5066-predictor-645f7d7b85-fsdjq |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.42:8643/healthz": dial tcp 10.132.0.42:8643: connect: connection refused | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-965a2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-965a2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-0934b-74995788f-j8psd |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-965a2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-965a2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-f5066-5dd8b587f8-6xmdh |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-965a2-predictor-64fd5c48d4-rjkhz |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.33:8080: connect: connection refused |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-afa2d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-afa2d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-afa2d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-afa2d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-965a2-predictor-6df948c77-gjp88 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.43:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-afa2d-predictor-54c8dbf4d6-k86d2 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.35:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-965a2 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-965a2 |
InferenceServiceReady |
InferenceService [success-200-isvc-965a2] is Ready | |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-afa2d-predictor-75594848c5-cc7sx |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.44:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-965a2 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-965a2 |
InferenceServiceReady |
InferenceService [error-404-isvc-965a2] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-afa2d |
InferenceServiceReady |
InferenceService [success-200-isvc-afa2d] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-afa2d |
InferenceServiceReady |
InferenceService [error-404-isvc-afa2d] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-afa2d |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-965a2-5ddc4d9f8b |
SuccessfulCreate |
Created pod: switch-graph-965a2-5ddc4d9f8b-lpck6 | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-965a2 |
ScalingReplicaSet |
Scaled up replica set switch-graph-965a2-5ddc4d9f8b from 0 to 1 | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-afa2d |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
kubelet |
switch-graph-965a2-5ddc4d9f8b-lpck6 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-965a2-serving-cert" not found | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-965a2 |
InferenceGraphReady |
InferenceGraph [switch-graph-965a2] is Ready | |
kserve-ci-e2e-test |
kubelet |
switch-graph-965a2-5ddc4d9f8b-lpck6 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:41cb8bf9ff6c7df05622085595fdb4c1e1d3a571d74d4bdea37a3b72ad46799b" already present on machine | |
kserve-ci-e2e-test |
multus |
switch-graph-965a2-5ddc4d9f8b-lpck6 |
AddedInterface |
Add eth0 [10.134.0.36/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-965a2-5ddc4d9f8b-lpck6 |
Created |
Created container: switch-graph-965a2 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-965a2-5ddc4d9f8b-lpck6 |
Started |
Started container switch-graph-965a2 | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-afa2d |
ScalingReplicaSet |
Scaled up replica set splitter-graph-afa2d-86d55575d from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-afa2d-86d55575d |
SuccessfulCreate |
Created pod: splitter-graph-afa2d-86d55575d-mkw79 | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-afa2d |
InferenceGraphReady |
InferenceGraph [splitter-graph-afa2d] is Ready | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-afa2d-86d55575d-mkw79 |
Created |
Created container: splitter-graph-afa2d | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-afa2d-86d55575d-mkw79 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:41cb8bf9ff6c7df05622085595fdb4c1e1d3a571d74d4bdea37a3b72ad46799b" already present on machine | |
kserve-ci-e2e-test |
multus |
splitter-graph-afa2d-86d55575d-mkw79 |
AddedInterface |
Add eth0 [10.134.0.37/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-afa2d-86d55575d-mkw79 |
Started |
Started container splitter-graph-afa2d | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-965a2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-965a2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-afa2d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-afa2d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-afa2d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-afa2d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-965a2 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-965a2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-965a2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-965a2 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-afa2d |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-afa2d |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
splitter-graph-afa2d-86d55575d-mkw79 |
Killing |
Stopping container splitter-graph-afa2d | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-afa2d-predictor-75594848c5-cc7sx |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-afa2d-predictor-75594848c5-cc7sx |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-afa2d-predictor-54c8dbf4d6-k86d2 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-afa2d-predictor-54c8dbf4d6-k86d2 |
Killing |
Stopping container kube-rbac-proxy | |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-afa2d-86d55575d-mkw79 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-965a2-predictor-6df948c77-gjp88 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-965a2-predictor-64fd5c48d4-rjkhz |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-965a2-predictor-64fd5c48d4-rjkhz |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-965a2-predictor-6df948c77-gjp88 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
switch-graph-965a2-5ddc4d9f8b-lpck6 |
Killing |
Stopping container switch-graph-965a2 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-965a2-predictor-6df948c77-gjp88 |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.43:8643/healthz": dial tcp 10.132.0.43:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-965a2-5ddc4d9f8b-lpck6 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |