| Time | Namespace | Component | RelatedObject | Reason | Message |
|---|---|---|---|---|---|
kserve-ci-e2e-test |
error-404-isvc-92a61-predictor-5b8cf99c8c-75dxv |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-92a61-predictor-5b8cf99c8c-75dxv to ip-10-0-142-142.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-305f4-d67d47567-krvxn |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-305f4-d67d47567-krvxn to ip-10-0-132-244.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-305f4-predictor-c7fb94cc6-66jqf |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-305f4-predictor-c7fb94cc6-66jqf to ip-10-0-142-142.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-d7086-ff98df69b-m7n8z |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-d7086-ff98df69b-m7n8z to ip-10-0-132-244.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-999e1-5df54fbb4c-8s8kp |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-999e1-5df54fbb4c-8s8kp to ip-10-0-132-244.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-dfad7-5f5d97c8c5-9l9jn |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-dfad7-5f5d97c8c5-9l9jn to ip-10-0-132-244.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-32e32-predictor-5fb6b9c6c8-2hsv2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-32e32-predictor-5fb6b9c6c8-2hsv2 to ip-10-0-142-142.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-85420-predictor-8c694c98b-cnn9k |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-85420-predictor-8c694c98b-cnn9k to ip-10-0-142-142.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-dfad7-predictor-9f88c844f-hddzq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-dfad7-predictor-9f88c844f-hddzq to ip-10-0-142-142.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-32e32-56c9887cf-bgkh9 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-32e32-56c9887cf-bgkh9 to ip-10-0-132-244.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-85420-predictor-856b7d987-9rsjd |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-85420-predictor-856b7d987-9rsjd to ip-10-0-142-142.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-858c3-7798d67676-c5m7h |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-858c3-7798d67676-c5m7h to ip-10-0-132-244.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-32e32-predictor-98848c587-cfphf |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-32e32-predictor-98848c587-cfphf to ip-10-0-142-142.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-92a61-78458fdc5-fngpc |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-92a61-78458fdc5-fngpc to ip-10-0-132-244.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-85420-5d67fb78c6-cxj5w |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-85420-5d67fb78c6-cxj5w to ip-10-0-132-244.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-24ddf-599f97958c-svd5v |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-24ddf-599f97958c-svd5v to ip-10-0-132-244.ec2.internal | ||
kserve-ci-e2e-test |
model-chainer-7dc7f99b6d-qtdht |
Scheduled |
Successfully assigned kserve-ci-e2e-test/model-chainer-7dc7f99b6d-qtdht to ip-10-0-132-244.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-dfad7-predictor-dffd7f4cb-6ddvq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-dfad7-predictor-dffd7f4cb-6ddvq to ip-10-0-142-142.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-858c3-predictor-6b7b999659-kmlhq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-858c3-predictor-6b7b999659-kmlhq to ip-10-0-142-142.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-87d4d-predictor-779b4f5579-8cp6c |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-87d4d-predictor-779b4f5579-8cp6c to ip-10-0-142-142.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-24ddf-predictor-b8f9d9964-25s57 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-24ddf-predictor-b8f9d9964-25s57 to ip-10-0-142-142.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-1-predictor-7fbf85bc5d-kp6d5 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7fbf85bc5d-kp6d5 to ip-10-0-137-106.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-305f4-predictor-5d5987d4f-pz2z2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-305f4-predictor-5d5987d4f-pz2z2 to ip-10-0-142-142.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-d7086-predictor-6848586898-5cvmn |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-d7086-predictor-6848586898-5cvmn to ip-10-0-142-142.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-24ddf-predictor-665cbb8c8f-25wjw |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-24ddf-predictor-665cbb8c8f-25wjw to ip-10-0-142-142.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-999e1-predictor-57944785f4-q888m |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-999e1-predictor-57944785f4-q888m to ip-10-0-142-142.ec2.internal | ||
kserve-ci-e2e-test |
isvc-xgboost-graph-predictor-555fc69d56-blq59 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-blq59 to ip-10-0-137-106.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-92a61-predictor-f78d96bf8-l44xt |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-92a61-predictor-f78d96bf8-l44xt to ip-10-0-142-142.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-999e1-predictor-fd6f8745f-njpnw |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-999e1-predictor-fd6f8745f-njpnw to ip-10-0-142-142.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-87d4d-predictor-5dfdb8946b-9snv2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-87d4d-predictor-5dfdb8946b-9snv2 to ip-10-0-142-142.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-d7086-predictor-84cb8df759-4cmm2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-d7086-predictor-84cb8df759-4cmm2 to ip-10-0-142-142.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-858c3-predictor-845dd5dbd6-f4bmt |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-858c3-predictor-845dd5dbd6-f4bmt to ip-10-0-142-142.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-2-predictor-6664c8d89d-hd2nz |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6664c8d89d-hd2nz to ip-10-0-132-244.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-87d4d-6dfd87976f-kww7r |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-87d4d-6dfd87976f-kww7r to ip-10-0-132-244.ec2.internal | ||
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-85420-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-85420-predictor-8c694c98b from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-85420 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-85420": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-85420": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-85420-predictor-8c694c98b |
SuccessfulCreate |
Created pod: success-200-isvc-85420-predictor-8c694c98b-cnn9k | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-85420 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-85420": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-1-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-1-predictor-7fbf85bc5d from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-1-predictor-7fbf85bc5d |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-1-predictor-7fbf85bc5d-kp6d5 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-85420-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-85420-predictor-856b7d987 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-85420-predictor-856b7d987 |
SuccessfulCreate |
Created pod: error-404-isvc-85420-predictor-856b7d987-9rsjd | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-85420-predictor-8c694c98b-cnn9k |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-85420-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7fbf85bc5d-kp6d5 |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-2-predictor-6664c8d89d |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-2-predictor-6664c8d89d-hd2nz | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7fbf85bc5d-kp6d5 |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:9b7f9014cc3727e4fd95a945572d3f70017252d596bd8941a1887e4a1514875c" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7fbf85bc5d-kp6d5 |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
multus |
success-200-isvc-85420-predictor-8c694c98b-cnn9k |
AddedInterface |
Add eth0 [10.134.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-85420-predictor-8c694c98b-cnn9k |
Pulling |
Pulling image "quay.io/opendatahub/success-200-isvc:odh-pr-1703" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-85420-predictor-856b7d987-9rsjd |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-85420-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
isvc-xgboost-graph-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-xgboost-graph-predictor-555fc69d56 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6664c8d89d-hd2nz |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "isvc-sklearn-graph-2-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-xgboost-graph-predictor-555fc69d56 |
SuccessfulCreate |
Created pod: isvc-xgboost-graph-predictor-555fc69d56-blq59 | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-1-predictor-7fbf85bc5d-kp6d5 |
AddedInterface |
Add eth0 [10.133.0.23/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-blq59 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "isvc-xgboost-graph-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-2-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-2-predictor-6664c8d89d from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-blq59 |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:9b7f9014cc3727e4fd95a945572d3f70017252d596bd8941a1887e4a1514875c" already present on machine | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-2-predictor-6664c8d89d-hd2nz |
AddedInterface |
Add eth0 [10.132.0.19/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-85420-predictor-856b7d987-9rsjd |
Pulling |
Pulling image "quay.io/opendatahub/error-404-isvc:odh-pr-1703" | |
kserve-ci-e2e-test |
multus |
error-404-isvc-85420-predictor-856b7d987-9rsjd |
AddedInterface |
Add eth0 [10.134.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-blq59 |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
multus |
isvc-xgboost-graph-predictor-555fc69d56-blq59 |
AddedInterface |
Add eth0 [10.133.0.24/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6664c8d89d-hd2nz |
Pulling |
Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:9b7f9014cc3727e4fd95a945572d3f70017252d596bd8941a1887e4a1514875c" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-blq59 |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6664c8d89d-hd2nz |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:9b7f9014cc3727e4fd95a945572d3f70017252d596bd8941a1887e4a1514875c" in 3.071s (3.071s including waiting). Image size: 300633537 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7fbf85bc5d-kp6d5 |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1703" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6664c8d89d-hd2nz |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6664c8d89d-hd2nz |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-blq59 |
Pulling |
Pulling image "kserve/xgbserver:latest" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6664c8d89d-hd2nz |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1703" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-85420-predictor-856b7d987-9rsjd |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-85420-predictor-856b7d987-9rsjd |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-85420-predictor-8c694c98b-cnn9k |
Pulled |
Successfully pulled image "quay.io/opendatahub/success-200-isvc:odh-pr-1703" in 15.661s (15.662s including waiting). Image size: 1335633000 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-85420-predictor-856b7d987-9rsjd |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-85420-predictor-856b7d987-9rsjd |
Pulled |
Successfully pulled image "quay.io/opendatahub/error-404-isvc:odh-pr-1703" in 15.28s (15.28s including waiting). Image size: 1334626402 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-85420-predictor-8c694c98b-cnn9k |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-85420-predictor-8c694c98b-cnn9k |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-85420-predictor-8c694c98b-cnn9k |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-85420-predictor-856b7d987-9rsjd |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.558s (2.558s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-85420-predictor-8c694c98b-cnn9k |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.548s (2.548s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-85420-predictor-856b7d987-9rsjd |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-85420-predictor-8c694c98b-cnn9k |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-85420-predictor-856b7d987-9rsjd |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-85420-predictor-8c694c98b-cnn9k |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6664c8d89d-hd2nz |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6664c8d89d-hd2nz |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6664c8d89d-hd2nz |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1703" in 12.746s (12.746s including waiting). Image size: 1555746730 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6664c8d89d-hd2nz |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6664c8d89d-hd2nz |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.228s (2.228s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6664c8d89d-hd2nz |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6664c8d89d-hd2nz |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7fbf85bc5d-kp6d5 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7fbf85bc5d-kp6d5 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7fbf85bc5d-kp6d5 |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7fbf85bc5d-kp6d5 |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1703" in 22.976s (22.976s including waiting). Image size: 1555746730 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-blq59 |
Pulled |
Successfully pulled image "kserve/xgbserver:latest" in 22.386s (22.386s including waiting). Image size: 1305293053 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-blq59 |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-blq59 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-blq59 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-blq59 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7fbf85bc5d-kp6d5 |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.968s (2.968s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7fbf85bc5d-kp6d5 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7fbf85bc5d-kp6d5 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-blq59 |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.567s (2.567s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-blq59 |
Created |
Created container: kube-rbac-proxy | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-85420-predictor-8c694c98b-cnn9k |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.32:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-85420-predictor-856b7d987-9rsjd |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.33:8080: connect: connection refused |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-85420-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-85420-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-85420-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-85420-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-85420 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-85420 |
InferenceServiceReady |
InferenceService [error-404-isvc-85420] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-85420 |
InferenceServiceReady |
InferenceService [success-200-isvc-85420] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-85420 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x7) | kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-blq59 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.24:8080: connect: connection refused |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-85420 |
InferenceGraphReady |
InferenceGraph [switch-graph-85420] is Ready | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-85420 |
UpdateFailed |
Failed to update status for InferenceGraph "switch-graph-85420": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "switch-graph-85420": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-85420 |
ScalingReplicaSet |
Scaled up replica set switch-graph-85420-5d67fb78c6 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-85420-5d67fb78c6 |
SuccessfulCreate |
Created pod: switch-graph-85420-5d67fb78c6-cxj5w | |
kserve-ci-e2e-test |
kubelet |
switch-graph-85420-5d67fb78c6-cxj5w |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-85420-serving-cert" not found | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-85420 |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "switch-graph-85420": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
switch-graph-85420-5d67fb78c6-cxj5w |
Pulling |
Pulling image "quay.io/opendatahub/kserve-router@sha256:a8d64b572a976cc85b28488a3ef8f7d50ce28ddb75186c1859612e88d6dba080" | |
kserve-ci-e2e-test |
multus |
switch-graph-85420-5d67fb78c6-cxj5w |
AddedInterface |
Add eth0 [10.132.0.20/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-85420-5d67fb78c6-cxj5w |
Created |
Created container: switch-graph-85420 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-85420-5d67fb78c6-cxj5w |
Started |
Started container switch-graph-85420 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-85420-5d67fb78c6-cxj5w |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-router@sha256:a8d64b572a976cc85b28488a3ef8f7d50ce28ddb75186c1859612e88d6dba080" in 2.119s (2.119s including waiting). Image size: 218071146 bytes. | |
| (x8) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7fbf85bc5d-kp6d5 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.23:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
InferenceServiceReady |
InferenceService [isvc-xgboost-graph] is Ready | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-87d4d-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-87d4d-predictor-5dfdb8946b from 0 to 1 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-85420-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-85420-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-85420-predictor-8c694c98b-cnn9k |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
switch-graph-85420-5d67fb78c6-cxj5w |
Killing |
Stopping container switch-graph-85420 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-85420-predictor-8c694c98b-cnn9k |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-87d4d |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-87d4d": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-87d4d": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-87d4d |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-87d4d": the object has been modified; please apply your changes to the latest version and try again | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-85420-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
multus |
error-404-isvc-87d4d-predictor-779b4f5579-8cp6c |
AddedInterface |
Add eth0 [10.134.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-87d4d-predictor-779b4f5579-8cp6c |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-87d4d-predictor-779b4f5579-8cp6c |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-87d4d-predictor-779b4f5579-8cp6c |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-87d4d-predictor-779b4f5579-8cp6c |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-87d4d-predictor-779b4f5579 |
SuccessfulCreate |
Created pod: error-404-isvc-87d4d-predictor-779b4f5579-8cp6c | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-87d4d-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-87d4d-predictor-779b4f5579 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-87d4d-predictor-779b4f5579-8cp6c |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1703" already present on machine | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-85420-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-85420-predictor-856b7d987-9rsjd |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-85420-predictor-8c694c98b-cnn9k |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.32:8643/healthz": dial tcp 10.134.0.32:8643: connect: connection refused | |
kserve-ci-e2e-test |
multus |
success-200-isvc-87d4d-predictor-5dfdb8946b-9snv2 |
AddedInterface |
Add eth0 [10.134.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-87d4d |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-87d4d": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-87d4d": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-87d4d |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-87d4d": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-85420-predictor-856b7d987-9rsjd |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.33:8643/healthz": dial tcp 10.134.0.33:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-87d4d-predictor-5dfdb8946b-9snv2 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1703" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-87d4d-predictor-5dfdb8946b-9snv2 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-87d4d-predictor-5dfdb8946b-9snv2 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-87d4d-predictor-5dfdb8946b-9snv2 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-87d4d-predictor-5dfdb8946b-9snv2 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-87d4d-predictor-5dfdb8946b-9snv2 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-87d4d-predictor-779b4f5579-8cp6c |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-87d4d-predictor-5dfdb8946b |
SuccessfulCreate |
Created pod: success-200-isvc-87d4d-predictor-5dfdb8946b-9snv2 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-85420-predictor-856b7d987-9rsjd |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-2] is Ready | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x8) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-1] is Ready | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-85420-5d67fb78c6-cxj5w |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
InferenceGraphController |
model-chainer |
InferenceGraphReady |
InferenceGraph [model-chainer] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
model-chainer |
ScalingReplicaSet |
Scaled up replica set model-chainer-7dc7f99b6d from 0 to 1 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-87d4d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-87d4d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
replicaset-controller |
model-chainer-7dc7f99b6d |
SuccessfulCreate |
Created pod: model-chainer-7dc7f99b6d-qtdht | |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
model-chainer-7dc7f99b6d-qtdht |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:a8d64b572a976cc85b28488a3ef8f7d50ce28ddb75186c1859612e88d6dba080" already present on machine | |
kserve-ci-e2e-test |
kubelet |
model-chainer-7dc7f99b6d-qtdht |
Created |
Created container: model-chainer | |
kserve-ci-e2e-test |
kubelet |
model-chainer-7dc7f99b6d-qtdht |
Started |
Started container model-chainer | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
multus |
model-chainer-7dc7f99b6d-qtdht |
AddedInterface |
Add eth0 [10.132.0.21/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7fbf85bc5d-kp6d5 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6664c8d89d-hd2nz |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6664c8d89d-hd2nz |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7fbf85bc5d-kp6d5 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-dfad7-predictor-dffd7f4cb |
SuccessfulCreate |
Created pod: success-200-isvc-dfad7-predictor-dffd7f4cb-6ddvq | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-dfad7 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-dfad7": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
model-chainer-7dc7f99b6d-qtdht |
Killing |
Stopping container model-chainer | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-dfad7-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-dfad7-predictor-dffd7f4cb from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-dfad7-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-dfad7-predictor-9f88c844f from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-dfad7 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-dfad7": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-dfad7": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
multus |
success-200-isvc-dfad7-predictor-dffd7f4cb-6ddvq |
AddedInterface |
Add eth0 [10.134.0.36/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-dfad7-predictor-dffd7f4cb-6ddvq |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-dfad7-predictor-9f88c844f-hddzq |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-dfad7-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-blq59 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-dfad7-predictor-dffd7f4cb-6ddvq |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1703" already present on machine | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-dfad7-predictor-9f88c844f |
SuccessfulCreate |
Created pod: error-404-isvc-dfad7-predictor-9f88c844f-hddzq | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-dfad7-predictor-dffd7f4cb-6ddvq |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-dfad7-predictor-dffd7f4cb-6ddvq |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-blq59 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-dfad7-predictor-dffd7f4cb-6ddvq |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-dfad7-predictor-dffd7f4cb-6ddvq |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-dfad7-predictor-9f88c844f-hddzq |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-dfad7-predictor-9f88c844f-hddzq |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
| (x9) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6664c8d89d-hd2nz |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.19:8080: connect: connection refused |
kserve-ci-e2e-test |
multus |
error-404-isvc-dfad7-predictor-9f88c844f-hddzq |
AddedInterface |
Add eth0 [10.134.0.37/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-dfad7-predictor-9f88c844f-hddzq |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1703" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6664c8d89d-hd2nz |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.19:8643/healthz": dial tcp 10.132.0.19:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-dfad7-predictor-9f88c844f-hddzq |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-dfad7-predictor-9f88c844f-hddzq |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-dfad7-predictor-9f88c844f-hddzq |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7fbf85bc5d-kp6d5 |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.23:8643/healthz": dial tcp 10.133.0.23:8643: connect: connection refused | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-87d4d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-87d4d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-87d4d |
InferenceServiceReady |
InferenceService [success-200-isvc-87d4d] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-87d4d |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-87d4d |
InferenceServiceReady |
InferenceService [error-404-isvc-87d4d] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-87d4d |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
kubelet |
switch-graph-87d4d-6dfd87976f-kww7r |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:a8d64b572a976cc85b28488a3ef8f7d50ce28ddb75186c1859612e88d6dba080" already present on machine | |
kserve-ci-e2e-test |
multus |
switch-graph-87d4d-6dfd87976f-kww7r |
AddedInterface |
Add eth0 [10.132.0.22/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-87d4d |
ScalingReplicaSet |
Scaled up replica set switch-graph-87d4d-6dfd87976f from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-87d4d |
InferenceGraphReady |
InferenceGraph [switch-graph-87d4d] is Ready | |
kserve-ci-e2e-test |
kubelet |
switch-graph-87d4d-6dfd87976f-kww7r |
Created |
Created container: switch-graph-87d4d | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-87d4d-6dfd87976f |
SuccessfulCreate |
Created pod: switch-graph-87d4d-6dfd87976f-kww7r | |
kserve-ci-e2e-test |
kubelet |
switch-graph-87d4d-6dfd87976f-kww7r |
Started |
Started container switch-graph-87d4d | |
| (x6) | kserve-ci-e2e-test |
kubelet |
model-chainer-7dc7f99b6d-qtdht |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-dfad7-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-dfad7-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-dfad7-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-dfad7-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-87d4d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-87d4d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-87d4d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-87d4d |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-87d4d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-87d4d |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-dfad7-predictor-9f88c844f-hddzq |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.37:8080: connect: connection refused |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-dfad7 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-dfad7 |
InferenceServiceReady |
InferenceService [success-200-isvc-dfad7] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-dfad7 |
InferenceServiceReady |
InferenceService [error-404-isvc-dfad7] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-dfad7 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-87d4d |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-87d4d |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-dfad7-5f5d97c8c5 |
SuccessfulCreate |
Created pod: sequence-graph-dfad7-5f5d97c8c5-9l9jn | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-dfad7 |
UpdateFailed |
Failed to update status for InferenceGraph "sequence-graph-dfad7": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "sequence-graph-dfad7": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-dfad7 |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "sequence-graph-dfad7": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-dfad7 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-dfad7-5f5d97c8c5 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-dfad7 |
InferenceGraphReady |
InferenceGraph [sequence-graph-dfad7] is Ready | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-dfad7-5f5d97c8c5-9l9jn |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:a8d64b572a976cc85b28488a3ef8f7d50ce28ddb75186c1859612e88d6dba080" already present on machine | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-dfad7-5f5d97c8c5-9l9jn |
Created |
Created container: sequence-graph-dfad7 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-dfad7-5f5d97c8c5-9l9jn |
Started |
Started container sequence-graph-dfad7 | |
kserve-ci-e2e-test |
multus |
sequence-graph-dfad7-5f5d97c8c5-9l9jn |
AddedInterface |
Add eth0 [10.132.0.23/23] from ovn-kubernetes | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-dfad7-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-dfad7-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-dfad7-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-dfad7-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-dfad7 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-dfad7 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-305f4 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-305f4": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-305f4": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-305f4-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-305f4-predictor-c7fb94cc6 from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-305f4 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-305f4": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
switch-graph-87d4d-6dfd87976f-kww7r |
Killing |
Stopping container switch-graph-87d4d | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-305f4-predictor-c7fb94cc6 |
SuccessfulCreate |
Created pod: success-200-isvc-305f4-predictor-c7fb94cc6-66jqf | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-87d4d-predictor-5dfdb8946b-9snv2 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-87d4d-predictor-5dfdb8946b-9snv2 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-305f4-predictor-5d5987d4f-pz2z2 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-305f4-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-87d4d-predictor-779b4f5579-8cp6c |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-305f4-predictor-c7fb94cc6-66jqf |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-305f4-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-305f4-predictor-5d5987d4f |
SuccessfulCreate |
Created pod: error-404-isvc-305f4-predictor-5d5987d4f-pz2z2 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-305f4 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-305f4": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-305f4": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-305f4 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-305f4": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-305f4-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-305f4-predictor-5d5987d4f from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-87d4d-predictor-779b4f5579-8cp6c |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-305f4-predictor-c7fb94cc6-66jqf |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-305f4-predictor-c7fb94cc6-66jqf |
Started |
Started container kserve-container | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-87d4d-predictor-5dfdb8946b-9snv2 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.34:8080: connect: connection refused |
kserve-ci-e2e-test |
multus |
success-200-isvc-305f4-predictor-c7fb94cc6-66jqf |
AddedInterface |
Add eth0 [10.134.0.38/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-305f4-predictor-5d5987d4f-pz2z2 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-305f4-predictor-c7fb94cc6-66jqf |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1703" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-305f4-predictor-5d5987d4f-pz2z2 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-87d4d-predictor-5dfdb8946b-9snv2 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.34:8643/healthz": dial tcp 10.134.0.34:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-305f4-predictor-5d5987d4f-pz2z2 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1703" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-305f4-predictor-5d5987d4f-pz2z2 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-305f4-predictor-5d5987d4f-pz2z2 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-305f4-predictor-c7fb94cc6-66jqf |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-305f4-predictor-5d5987d4f-pz2z2 |
AddedInterface |
Add eth0 [10.134.0.39/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-305f4-predictor-5d5987d4f-pz2z2 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-305f4-predictor-c7fb94cc6-66jqf |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-305f4-predictor-c7fb94cc6-66jqf |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-87d4d-predictor-779b4f5579-8cp6c |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.35:8643/healthz": dial tcp 10.134.0.35:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-87d4d-predictor-779b4f5579-8cp6c |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.35:8080: connect: connection refused |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-305f4-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-305f4-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-87d4d-6dfd87976f-kww7r |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-305f4-predictor-5d5987d4f-pz2z2 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.39:8080: connect: connection refused |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-24ddf-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-24ddf-predictor-b8f9d9964 from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-24ddf-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-24ddf-predictor-665cbb8c8f from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-dfad7-predictor-9f88c844f-hddzq |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-24ddf-predictor-b8f9d9964 |
SuccessfulCreate |
Created pod: success-200-isvc-24ddf-predictor-b8f9d9964-25s57 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-dfad7-predictor-dffd7f4cb-6ddvq |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-dfad7-predictor-dffd7f4cb-6ddvq |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-dfad7-5f5d97c8c5-9l9jn |
Killing |
Stopping container sequence-graph-dfad7 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-dfad7-predictor-9f88c844f-hddzq |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-24ddf-predictor-665cbb8c8f-25wjw |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-24ddf-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-24ddf-predictor-b8f9d9964-25s57 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-24ddf-predictor-b8f9d9964-25s57 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-24ddf-predictor-665cbb8c8f |
SuccessfulCreate |
Created pod: error-404-isvc-24ddf-predictor-665cbb8c8f-25wjw | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-24ddf-predictor-b8f9d9964-25s57 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-24ddf-predictor-b8f9d9964-25s57 |
AddedInterface |
Add eth0 [10.134.0.40/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-24ddf-predictor-b8f9d9964-25s57 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1703" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-24ddf-predictor-b8f9d9964-25s57 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-24ddf-predictor-b8f9d9964-25s57 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-24ddf |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-24ddf": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-24ddf |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-24ddf": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-24ddf": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-24ddf-predictor-665cbb8c8f-25wjw |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-24ddf-predictor-665cbb8c8f-25wjw |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-24ddf-predictor-665cbb8c8f-25wjw |
AddedInterface |
Add eth0 [10.134.0.41/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-dfad7-predictor-dffd7f4cb-6ddvq |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.36:8643/healthz": dial tcp 10.134.0.36:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-24ddf-predictor-665cbb8c8f-25wjw |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-24ddf-predictor-665cbb8c8f-25wjw |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-24ddf-predictor-665cbb8c8f-25wjw |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1703" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-24ddf-predictor-665cbb8c8f-25wjw |
Started |
Started container kserve-container | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-dfad7-predictor-dffd7f4cb-6ddvq |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.36:8080: connect: connection refused |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-305f4-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-305f4-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-305f4-predictor-c7fb94cc6-66jqf |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.38:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-305f4 |
InferenceServiceReady |
InferenceService [error-404-isvc-305f4] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-305f4 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-305f4 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-305f4 |
InferenceServiceReady |
InferenceService [success-200-isvc-305f4] is Ready | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-dfad7-5f5d97c8c5-9l9jn |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-24ddf-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-24ddf-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-24ddf-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-305f4-d67d47567 |
SuccessfulCreate |
Created pod: ensemble-graph-305f4-d67d47567-krvxn | |
| (x2) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-305f4-d67d47567-krvxn |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "ensemble-graph-305f4-serving-cert" not found |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-24ddf-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-305f4 |
InferenceGraphReady |
InferenceGraph [ensemble-graph-305f4] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-305f4 |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-305f4-d67d47567 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-305f4-d67d47567-krvxn |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:a8d64b572a976cc85b28488a3ef8f7d50ce28ddb75186c1859612e88d6dba080" already present on machine | |
kserve-ci-e2e-test |
multus |
ensemble-graph-305f4-d67d47567-krvxn |
AddedInterface |
Add eth0 [10.132.0.24/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-305f4-d67d47567-krvxn |
Created |
Created container: ensemble-graph-305f4 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-305f4-d67d47567-krvxn |
Started |
Started container ensemble-graph-305f4 | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-305f4-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-305f4-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-305f4-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-305f4-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-24ddf-predictor-b8f9d9964-25s57 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.40:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-24ddf-predictor-665cbb8c8f-25wjw |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.41:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-858c3-predictor-845dd5dbd6-f4bmt |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1703" already present on machine | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-305f4-d67d47567-krvxn |
Killing |
Stopping container ensemble-graph-305f4 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-858c3-predictor-845dd5dbd6-f4bmt |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-858c3-predictor-845dd5dbd6-f4bmt |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-858c3 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-858c3": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-858c3": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-858c3 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-858c3": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-858c3 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-858c3": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-858c3-predictor-845dd5dbd6-f4bmt |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-858c3 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-858c3": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-858c3": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
multus |
success-200-isvc-858c3-predictor-845dd5dbd6-f4bmt |
AddedInterface |
Add eth0 [10.134.0.42/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-858c3-predictor-6b7b999659-kmlhq |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-858c3-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-305f4-predictor-c7fb94cc6-66jqf |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-858c3-predictor-845dd5dbd6-f4bmt |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-858c3-predictor-845dd5dbd6-f4bmt |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-858c3-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-858c3-predictor-845dd5dbd6 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-858c3-predictor-845dd5dbd6 |
SuccessfulCreate |
Created pod: success-200-isvc-858c3-predictor-845dd5dbd6-f4bmt | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-858c3-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-858c3-predictor-6b7b999659 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-858c3-predictor-6b7b999659 |
SuccessfulCreate |
Created pod: error-404-isvc-858c3-predictor-6b7b999659-kmlhq | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-305f4-predictor-5d5987d4f-pz2z2 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-305f4-predictor-5d5987d4f-pz2z2 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-305f4-predictor-c7fb94cc6-66jqf |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-858c3-predictor-6b7b999659-kmlhq |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-858c3-predictor-6b7b999659-kmlhq |
AddedInterface |
Add eth0 [10.134.0.43/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-858c3-predictor-6b7b999659-kmlhq |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1703" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-858c3-predictor-6b7b999659-kmlhq |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-858c3-predictor-6b7b999659-kmlhq |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-858c3-predictor-6b7b999659-kmlhq |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-858c3-predictor-6b7b999659-kmlhq |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-305f4-predictor-c7fb94cc6-66jqf |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.38:8643/healthz": dial tcp 10.134.0.38:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-305f4-predictor-5d5987d4f-pz2z2 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.39:8643/healthz": dial tcp 10.134.0.39:8643: connect: connection refused | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-24ddf |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-24ddf |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-24ddf |
InferenceServiceReady |
InferenceService [error-404-isvc-24ddf] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-24ddf |
InferenceServiceReady |
InferenceService [success-200-isvc-24ddf] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-24ddf-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-24ddf-599f97958c |
SuccessfulCreate |
Created pod: sequence-graph-24ddf-599f97958c-svd5v | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-24ddf |
InferenceGraphReady |
InferenceGraph [sequence-graph-24ddf] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-24ddf |
ScalingReplicaSet |
Scaled up replica set sequence-graph-24ddf-599f97958c from 0 to 1 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-24ddf-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
sequence-graph-24ddf-599f97958c-svd5v |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:a8d64b572a976cc85b28488a3ef8f7d50ce28ddb75186c1859612e88d6dba080" already present on machine | |
kserve-ci-e2e-test |
multus |
sequence-graph-24ddf-599f97958c-svd5v |
AddedInterface |
Add eth0 [10.132.0.25/23] from ovn-kubernetes | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-24ddf-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-24ddf-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
sequence-graph-24ddf-599f97958c-svd5v |
Created |
Created container: sequence-graph-24ddf | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-24ddf-599f97958c-svd5v |
Started |
Started container sequence-graph-24ddf | |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-305f4-d67d47567-krvxn |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-92a61 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-92a61": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-92a61": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-24ddf-599f97958c-svd5v |
Killing |
Stopping container sequence-graph-24ddf | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-92a61-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-92a61-predictor-f78d96bf8 from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-92a61 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-92a61": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-92a61-predictor-5b8cf99c8c-75dxv |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-92a61-predictor-f78d96bf8-l44xt |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1703" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-24ddf-predictor-b8f9d9964-25s57 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-24ddf-predictor-b8f9d9964-25s57 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-92a61-predictor-5b8cf99c8c-75dxv |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-92a61-predictor-5b8cf99c8c-75dxv |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1703" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-92a61-predictor-5b8cf99c8c-75dxv |
AddedInterface |
Add eth0 [10.134.0.45/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-92a61-predictor-5b8cf99c8c-75dxv |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-24ddf-predictor-665cbb8c8f-25wjw |
Killing |
Stopping container kube-rbac-proxy | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-858c3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-858c3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-92a61-predictor-f78d96bf8 |
SuccessfulCreate |
Created pod: success-200-isvc-92a61-predictor-f78d96bf8-l44xt | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-92a61-predictor-f78d96bf8-l44xt |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-92a61-predictor-f78d96bf8-l44xt |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-92a61 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-92a61": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-92a61 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-92a61": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-92a61": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-92a61-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-92a61-predictor-5b8cf99c8c from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-92a61-predictor-5b8cf99c8c |
SuccessfulCreate |
Created pod: error-404-isvc-92a61-predictor-5b8cf99c8c-75dxv | |
kserve-ci-e2e-test |
multus |
success-200-isvc-92a61-predictor-f78d96bf8-l44xt |
AddedInterface |
Add eth0 [10.134.0.44/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-24ddf-predictor-665cbb8c8f-25wjw |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-92a61-predictor-5b8cf99c8c-75dxv |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-92a61-predictor-f78d96bf8-l44xt |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-92a61-predictor-5b8cf99c8c-75dxv |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-92a61-predictor-f78d96bf8-l44xt |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-92a61-predictor-f78d96bf8-l44xt |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-24ddf-predictor-b8f9d9964-25s57 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.40:8643/healthz": dial tcp 10.134.0.40:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-24ddf-predictor-665cbb8c8f-25wjw |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.41:8643/healthz": dial tcp 10.134.0.41:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-858c3-predictor-6b7b999659-kmlhq |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.43:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-858c3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-858c3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-858c3 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-858c3 |
InferenceServiceReady |
InferenceService [success-200-isvc-858c3] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-858c3 |
InferenceServiceReady |
InferenceService [error-404-isvc-858c3] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-858c3 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-24ddf-599f97958c-svd5v |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-92a61-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-92a61-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-858c3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-92a61-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-858c3 |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-858c3-7798d67676 from 0 to 1 | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-858c3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-858c3-7798d67676-c5m7h |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "ensemble-graph-858c3-serving-cert" not found | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-92a61-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-858c3 |
InferenceGraphReady |
InferenceGraph [ensemble-graph-858c3] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-858c3-7798d67676 |
SuccessfulCreate |
Created pod: ensemble-graph-858c3-7798d67676-c5m7h | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-858c3-7798d67676-c5m7h |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:a8d64b572a976cc85b28488a3ef8f7d50ce28ddb75186c1859612e88d6dba080" already present on machine | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-858c3-7798d67676-c5m7h |
Started |
Started container ensemble-graph-858c3 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-858c3-7798d67676-c5m7h |
Created |
Created container: ensemble-graph-858c3 | |
kserve-ci-e2e-test |
multus |
ensemble-graph-858c3-7798d67676-c5m7h |
AddedInterface |
Add eth0 [10.132.0.26/23] from ovn-kubernetes | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-858c3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-858c3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-92a61 |
InferenceServiceReady |
InferenceService [success-200-isvc-92a61] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-92a61 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-92a61 |
InferenceServiceReady |
InferenceService [error-404-isvc-92a61] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-92a61 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-92a61-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-92a61 |
InferenceGraphReady |
InferenceGraph [sequence-graph-92a61] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-858c3 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
sequence-graph-92a61-78458fdc5-fngpc |
Started |
Started container sequence-graph-92a61 | |
kserve-ci-e2e-test |
multus |
sequence-graph-92a61-78458fdc5-fngpc |
AddedInterface |
Add eth0 [10.132.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-92a61-78458fdc5-fngpc |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:a8d64b572a976cc85b28488a3ef8f7d50ce28ddb75186c1859612e88d6dba080" already present on machine | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-92a61-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-92a61-78458fdc5 |
SuccessfulCreate |
Created pod: sequence-graph-92a61-78458fdc5-fngpc | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-92a61 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-92a61-78458fdc5 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-92a61-78458fdc5-fngpc |
Created |
Created container: sequence-graph-92a61 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-858c3 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-92a61-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-92a61-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-92a61 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-92a61 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-858c3-7798d67676-c5m7h |
Killing |
Stopping container ensemble-graph-858c3 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-999e1-predictor-57944785f4-q888m |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-858c3-predictor-845dd5dbd6-f4bmt |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-999e1-predictor-fd6f8745f-njpnw |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-858c3-predictor-6b7b999659-kmlhq |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-858c3-predictor-6b7b999659-kmlhq |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-999e1 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-999e1": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-999e1": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-999e1-predictor-fd6f8745f-njpnw |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-999e1-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-999e1-predictor-57944785f4 from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-999e1 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-999e1": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-999e1 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-999e1": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-999e1": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
multus |
error-404-isvc-999e1-predictor-fd6f8745f-njpnw |
AddedInterface |
Add eth0 [10.134.0.47/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-999e1-predictor-fd6f8745f-njpnw |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1703" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-999e1-predictor-fd6f8745f-njpnw |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-999e1-predictor-57944785f4-q888m |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1703" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-858c3-predictor-845dd5dbd6-f4bmt |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-999e1-predictor-57944785f4-q888m |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-999e1-predictor-57944785f4-q888m |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-999e1-predictor-fd6f8745f-njpnw |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-999e1-predictor-fd6f8745f-njpnw |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-999e1 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-999e1": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-999e1-predictor-57944785f4 |
SuccessfulCreate |
Created pod: success-200-isvc-999e1-predictor-57944785f4-q888m | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-999e1-predictor-57944785f4-q888m |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-999e1-predictor-57944785f4-q888m |
AddedInterface |
Add eth0 [10.134.0.46/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-999e1-predictor-57944785f4-q888m |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-999e1-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-999e1-predictor-fd6f8745f from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-999e1-predictor-fd6f8745f |
SuccessfulCreate |
Created pod: error-404-isvc-999e1-predictor-fd6f8745f-njpnw | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-858c3-predictor-845dd5dbd6-f4bmt |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.42:8643/healthz": dial tcp 10.134.0.42:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-858c3-predictor-845dd5dbd6-f4bmt |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.42:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-858c3-predictor-6b7b999659-kmlhq |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.43:8080: i/o timeout | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-858c3-predictor-6b7b999659-kmlhq |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.43:8643/healthz": context deadline exceeded | |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-858c3-7798d67676-c5m7h |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-32e32-predictor-5fb6b9c6c8 |
SuccessfulCreate |
Created pod: success-200-isvc-32e32-predictor-5fb6b9c6c8-2hsv2 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-92a61-78458fdc5-fngpc |
Killing |
Stopping container sequence-graph-92a61 | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-32e32-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-32e32-predictor-5fb6b9c6c8 from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-32e32 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-32e32": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-32e32": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-32e32 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-32e32": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
multus |
success-200-isvc-32e32-predictor-5fb6b9c6c8-2hsv2 |
AddedInterface |
Add eth0 [10.134.0.48/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-92a61-predictor-f78d96bf8-l44xt |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-999e1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-32e32 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-32e32": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-32e32 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-32e32": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-32e32": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-999e1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-32e32-predictor-98848c587-cfphf |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1703" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-32e32-predictor-98848c587-cfphf |
AddedInterface |
Add eth0 [10.134.0.49/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-32e32-predictor-98848c587 |
SuccessfulCreate |
Created pod: error-404-isvc-32e32-predictor-98848c587-cfphf | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-32e32-predictor-5fb6b9c6c8-2hsv2 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-32e32-predictor-98848c587-cfphf |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-32e32-predictor-98848c587-cfphf |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-32e32-predictor-5fb6b9c6c8-2hsv2 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-92a61-predictor-5b8cf99c8c-75dxv |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-32e32-predictor-98848c587-cfphf |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-999e1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-32e32-predictor-98848c587-cfphf |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-32e32-predictor-5fb6b9c6c8-2hsv2 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1703" already present on machine | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-999e1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-32e32-predictor-98848c587-cfphf |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-32e32-predictor-5fb6b9c6c8-2hsv2 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-32e32-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-32e32-predictor-98848c587 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-92a61-predictor-5b8cf99c8c-75dxv |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-92a61-predictor-f78d96bf8-l44xt |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-32e32-predictor-5fb6b9c6c8-2hsv2 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-32e32-predictor-5fb6b9c6c8-2hsv2 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-92a61-predictor-f78d96bf8-l44xt |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.44:8643/healthz": dial tcp 10.134.0.44:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-92a61-predictor-f78d96bf8-l44xt |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.44:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-92a61-predictor-5b8cf99c8c-75dxv |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.45:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-92a61-predictor-5b8cf99c8c-75dxv |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.45:8643/healthz": dial tcp 10.134.0.45:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-999e1-predictor-57944785f4-q888m |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.46:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-999e1-predictor-fd6f8745f-njpnw |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.47:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-999e1 |
InferenceServiceReady |
InferenceService [success-200-isvc-999e1] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-999e1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-999e1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-999e1 |
InferenceServiceReady |
InferenceService [error-404-isvc-999e1] is Ready | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-92a61-78458fdc5-fngpc |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-32e32-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-32e32-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-999e1 |
ScalingReplicaSet |
Scaled up replica set splitter-graph-999e1-5df54fbb4c from 0 to 1 | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-999e1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-999e1 |
InferenceGraphReady |
InferenceGraph [splitter-graph-999e1] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-999e1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-999e1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-32e32-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-999e1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-32e32-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-999e1-5df54fbb4c |
SuccessfulCreate |
Created pod: splitter-graph-999e1-5df54fbb4c-8s8kp | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-999e1-5df54fbb4c-8s8kp |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "splitter-graph-999e1-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-999e1-5df54fbb4c-8s8kp |
Started |
Started container splitter-graph-999e1 | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-999e1-5df54fbb4c-8s8kp |
Created |
Created container: splitter-graph-999e1 | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-999e1-5df54fbb4c-8s8kp |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:a8d64b572a976cc85b28488a3ef8f7d50ce28ddb75186c1859612e88d6dba080" already present on machine | |
kserve-ci-e2e-test |
multus |
splitter-graph-999e1-5df54fbb4c-8s8kp |
AddedInterface |
Add eth0 [10.132.0.28/23] from ovn-kubernetes | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-32e32-predictor-5fb6b9c6c8-2hsv2 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.48:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-32e32-predictor-98848c587-cfphf |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.49:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d7086 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-d7086": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d7086 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-d7086": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-999e1-predictor-57944785f4-q888m |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d7086-predictor-6848586898-5cvmn |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-d7086-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-999e1-5df54fbb4c-8s8kp |
Killing |
Stopping container splitter-graph-999e1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d7086 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-d7086": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-d7086": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-d7086-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-d7086-predictor-84cb8df759 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-d7086-predictor-84cb8df759 |
SuccessfulCreate |
Created pod: error-404-isvc-d7086-predictor-84cb8df759-4cmm2 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d7086-predictor-84cb8df759-4cmm2 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-d7086-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-999e1-predictor-fd6f8745f-njpnw |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-999e1-predictor-fd6f8745f-njpnw |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-999e1-predictor-57944785f4-q888m |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d7086 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-d7086": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-d7086": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-d7086-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-d7086-predictor-6848586898 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-d7086-predictor-6848586898 |
SuccessfulCreate |
Created pod: success-200-isvc-d7086-predictor-6848586898-5cvmn | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d7086-predictor-84cb8df759-4cmm2 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d7086-predictor-84cb8df759-4cmm2 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-d7086-predictor-84cb8df759-4cmm2 |
AddedInterface |
Add eth0 [10.134.0.51/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d7086-predictor-6848586898-5cvmn |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1703" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d7086-predictor-6848586898-5cvmn |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d7086-predictor-6848586898-5cvmn |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
success-200-isvc-d7086-predictor-6848586898-5cvmn |
AddedInterface |
Add eth0 [10.134.0.50/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d7086-predictor-84cb8df759-4cmm2 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d7086-predictor-84cb8df759-4cmm2 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1703" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d7086-predictor-84cb8df759-4cmm2 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d7086-predictor-84cb8df759-4cmm2 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d7086-predictor-6848586898-5cvmn |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d7086-predictor-6848586898-5cvmn |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d7086-predictor-6848586898-5cvmn |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-999e1-predictor-57944785f4-q888m |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.46:8643/healthz": dial tcp 10.134.0.46:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-999e1-predictor-fd6f8745f-njpnw |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.47:8643/healthz": dial tcp 10.134.0.47:8643: connect: connection refused | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-32e32 |
InferenceServiceReady |
InferenceService [success-200-isvc-32e32] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-32e32 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-32e32 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-32e32 |
InferenceServiceReady |
InferenceService [error-404-isvc-32e32] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-32e32 |
ScalingReplicaSet |
Scaled up replica set switch-graph-32e32-56c9887cf from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-32e32-56c9887cf |
SuccessfulCreate |
Created pod: switch-graph-32e32-56c9887cf-bgkh9 | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-32e32 |
UpdateFailed |
Failed to update status for InferenceGraph "switch-graph-32e32": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "switch-graph-32e32": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-32e32 |
InferenceGraphReady |
InferenceGraph [switch-graph-32e32] is Ready | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-32e32-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-32e32-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-32e32 |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "switch-graph-32e32": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
switch-graph-32e32-56c9887cf-bgkh9 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-32e32-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
switch-graph-32e32-56c9887cf-bgkh9 |
Created |
Created container: switch-graph-32e32 | |
kserve-ci-e2e-test |
multus |
switch-graph-32e32-56c9887cf-bgkh9 |
AddedInterface |
Add eth0 [10.132.0.29/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-32e32-56c9887cf-bgkh9 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:a8d64b572a976cc85b28488a3ef8f7d50ce28ddb75186c1859612e88d6dba080" already present on machine | |
kserve-ci-e2e-test |
kubelet |
switch-graph-32e32-56c9887cf-bgkh9 |
Started |
Started container switch-graph-32e32 | |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-999e1-5df54fbb4c-8s8kp |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-32e32-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-32e32-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-d7086-predictor-6848586898-5cvmn |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.50:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-d7086-predictor-84cb8df759-4cmm2 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.51:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d7086-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d7086-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d7086-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d7086-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d7086 |
InferenceServiceReady |
InferenceService [success-200-isvc-d7086] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d7086 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-32e32 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d7086 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-32e32 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d7086 |
InferenceServiceReady |
InferenceService [error-404-isvc-d7086] is Ready | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-d7086-ff98df69b-m7n8z |
Created |
Created container: splitter-graph-d7086 | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-d7086 |
InferenceGraphReady |
InferenceGraph [splitter-graph-d7086] is Ready | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-d7086-ff98df69b-m7n8z |
Started |
Started container splitter-graph-d7086 | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-d7086-ff98df69b |
SuccessfulCreate |
Created pod: splitter-graph-d7086-ff98df69b-m7n8z | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-d7086-ff98df69b-m7n8z |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:a8d64b572a976cc85b28488a3ef8f7d50ce28ddb75186c1859612e88d6dba080" already present on machine | |
kserve-ci-e2e-test |
multus |
splitter-graph-d7086-ff98df69b-m7n8z |
AddedInterface |
Add eth0 [10.132.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-d7086 |
ScalingReplicaSet |
Scaled up replica set splitter-graph-d7086-ff98df69b from 0 to 1 | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d7086-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-d7086 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d7086-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-d7086 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d7086-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d7086-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d7086-predictor-84cb8df759-4cmm2 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d7086-predictor-84cb8df759-4cmm2 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-d7086-ff98df69b-m7n8z |
Killing |
Stopping container splitter-graph-d7086 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d7086-predictor-6848586898-5cvmn |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d7086-predictor-6848586898-5cvmn |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d7086-predictor-6848586898-5cvmn |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.50:8643/healthz": context deadline exceeded | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d7086-predictor-6848586898-5cvmn |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.50:8080: i/o timeout | |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-d7086-ff98df69b-m7n8z |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
switch-graph-32e32-56c9887cf-bgkh9 |
Killing |
Stopping container switch-graph-32e32 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-32e32-predictor-5fb6b9c6c8-2hsv2 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-32e32-predictor-5fb6b9c6c8-2hsv2 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-32e32-predictor-98848c587-cfphf |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-32e32-predictor-98848c587-cfphf |
Killing |
Stopping container kube-rbac-proxy | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-32e32-56c9887cf-bgkh9 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |