| Time | Namespace | Component | RelatedObject | Reason | Message |
|---|---|---|---|---|---|
kserve-ci-e2e-test |
error-404-isvc-1e4e7-predictor-86f4b8b95d-glqzn |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-1e4e7-predictor-86f4b8b95d-glqzn to ip-10-0-133-107.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-8142b-d9d5fcc5c-79qkv |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-8142b-d9d5fcc5c-79qkv to ip-10-0-133-107.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-5ec4c-58bffd5c64-54zd4 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-5ec4c-58bffd5c64-54zd4 to ip-10-0-133-107.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-c397f-predictor-699fb4686-ws4j5 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-c397f-predictor-699fb4686-ws4j5 to ip-10-0-133-107.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-dcaf3-predictor-5fdbd86d9b-xb9bl |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-dcaf3-predictor-5fdbd86d9b-xb9bl to ip-10-0-133-107.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-892d5-predictor-6659f5749d-6qmvb |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-892d5-predictor-6659f5749d-6qmvb to ip-10-0-133-107.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-1-predictor-6f7b8d4cb-9t97d |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f7b8d4cb-9t97d to ip-10-0-139-254.ec2.internal | ||
kserve-ci-e2e-test |
isvc-xgboost-graph-predictor-555fc69d56-xkt8s |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xkt8s to ip-10-0-139-254.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-4a129-predictor-579fd87984-zc225 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-4a129-predictor-579fd87984-zc225 to ip-10-0-133-107.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-1fda1-predictor-c86ccb888-n4lkk |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-1fda1-predictor-c86ccb888-n4lkk to ip-10-0-133-107.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-1175d-68b6c76bfd-vkdb6 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-1175d-68b6c76bfd-vkdb6 to ip-10-0-133-107.ec2.internal | ||
kserve-ci-e2e-test |
model-chainer-597fd9c774-rhswc |
Scheduled |
Successfully assigned kserve-ci-e2e-test/model-chainer-597fd9c774-rhswc to ip-10-0-133-107.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-4a129-8fdcd5874-qjxjz |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-4a129-8fdcd5874-qjxjz to ip-10-0-133-107.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-dcaf3-predictor-5b5b4dfc-f5n25 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-dcaf3-predictor-5b5b4dfc-f5n25 to ip-10-0-134-158.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-c397f-predictor-84cc68fd6c-lzhj9 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-c397f-predictor-84cc68fd6c-lzhj9 to ip-10-0-134-158.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-1fda1-756f77b956-96xnd |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-1fda1-756f77b956-96xnd to ip-10-0-133-107.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-9819a-predictor-69df5dcdcf-bn5vz |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-9819a-predictor-69df5dcdcf-bn5vz to ip-10-0-134-158.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-892d5-predictor-b75d9549f-ffcm2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-892d5-predictor-b75d9549f-ffcm2 to ip-10-0-134-158.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-8142b-predictor-7948b5d7c6-zd5ng |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-8142b-predictor-7948b5d7c6-zd5ng to ip-10-0-133-107.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-1e4e7-6f5ffcc466-vpjqm |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-1e4e7-6f5ffcc466-vpjqm to ip-10-0-133-107.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-c397f-66d6dcb945-6dwwc |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-c397f-66d6dcb945-6dwwc to ip-10-0-133-107.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-2-predictor-ffc456649-z8txx |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-ffc456649-z8txx to ip-10-0-139-254.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-dcaf3-8fd969546-vzrkj |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-dcaf3-8fd969546-vzrkj to ip-10-0-133-107.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-892d5-75f7f9d99c-4gtmq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-892d5-75f7f9d99c-4gtmq to ip-10-0-133-107.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-9819a-5b6b565754-mn9bp |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-9819a-5b6b565754-mn9bp to ip-10-0-133-107.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-1175d-predictor-c88c69555-c5nkn |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-1175d-predictor-c88c69555-c5nkn to ip-10-0-134-158.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-1e4e7-predictor-76564f499-76j8q |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-1e4e7-predictor-76564f499-76j8q to ip-10-0-134-158.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-1175d-predictor-6784b69858-d8x9r |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-1175d-predictor-6784b69858-d8x9r to ip-10-0-133-107.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-1fda1-predictor-67c979686f-cnskz |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-1fda1-predictor-67c979686f-cnskz to ip-10-0-134-158.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-8142b-predictor-79bffc4977-htq8k |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-8142b-predictor-79bffc4977-htq8k to ip-10-0-134-158.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-5ec4c-predictor-5d6f7b484c-9z4xt |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-5ec4c-predictor-5d6f7b484c-9z4xt to ip-10-0-134-158.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-9819a-predictor-75b9f568db-gpdmf |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-9819a-predictor-75b9f568db-gpdmf to ip-10-0-133-107.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-5ec4c-predictor-68b6c44d66-hmwnv |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-5ec4c-predictor-68b6c44d66-hmwnv to ip-10-0-133-107.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-4a129-predictor-5c488f9bbf-ljp4w |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-4a129-predictor-5c488f9bbf-ljp4w to ip-10-0-134-158.ec2.internal | ||
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-1-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-1-predictor-6f7b8d4cb from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
isvc-xgboost-graph-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-xgboost-graph-predictor-555fc69d56 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-2-predictor-ffc456649 |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-2-predictor-ffc456649-z8txx | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-4a129-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-4a129-predictor-5c488f9bbf from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-4a129-predictor-5c488f9bbf |
SuccessfulCreate |
Created pod: success-200-isvc-4a129-predictor-5c488f9bbf-ljp4w | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-4a129-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-4a129-predictor-579fd87984 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-4a129-predictor-579fd87984 |
SuccessfulCreate |
Created pod: error-404-isvc-4a129-predictor-579fd87984-zc225 | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-1-predictor-6f7b8d4cb-9t97d |
AddedInterface |
Add eth0 [10.132.0.23/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6f7b8d4cb-9t97d |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6f7b8d4cb-9t97d |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-1-predictor-6f7b8d4cb |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-1-predictor-6f7b8d4cb-9t97d | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-2-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-2-predictor-ffc456649 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-xgboost-graph-predictor-555fc69d56 |
SuccessfulCreate |
Created pod: isvc-xgboost-graph-predictor-555fc69d56-xkt8s | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6f7b8d4cb-9t97d |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:051af6a7c6b8b08a9503c479397f58819d99aa2694c94c5895c362bcc590737e" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a129-predictor-579fd87984-zc225 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-4a129-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
multus |
error-404-isvc-4a129-predictor-579fd87984-zc225 |
AddedInterface |
Add eth0 [10.134.0.36/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a129-predictor-579fd87984-zc225 |
Pulling |
Pulling image "quay.io/opendatahub/error-404-isvc:odh-pr-1928" | |
kserve-ci-e2e-test |
multus |
isvc-xgboost-graph-predictor-555fc69d56-xkt8s |
AddedInterface |
Add eth0 [10.132.0.24/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xkt8s |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:051af6a7c6b8b08a9503c479397f58819d99aa2694c94c5895c362bcc590737e" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a129-predictor-5c488f9bbf-ljp4w |
FailedMount |
MountVolume.SetUp failed for volume "kube-rbac-proxy-sar-config" : failed to sync configmap cache: timed out waiting for the condition | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xkt8s |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xkt8s |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-ffc456649-z8txx |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "isvc-sklearn-graph-2-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-ffc456649-z8txx |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:051af6a7c6b8b08a9503c479397f58819d99aa2694c94c5895c362bcc590737e" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-ffc456649-z8txx |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-2-predictor-ffc456649-z8txx |
AddedInterface |
Add eth0 [10.132.0.25/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-ffc456649-z8txx |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
multus |
success-200-isvc-4a129-predictor-5c488f9bbf-ljp4w |
AddedInterface |
Add eth0 [10.133.0.16/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a129-predictor-5c488f9bbf-ljp4w |
Pulling |
Pulling image "quay.io/opendatahub/success-200-isvc:odh-pr-1928" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6f7b8d4cb-9t97d |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1928" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-ffc456649-z8txx |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1928" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xkt8s |
Pulling |
Pulling image "kserve/xgbserver:latest" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a129-predictor-579fd87984-zc225 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a129-predictor-5c488f9bbf-ljp4w |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a129-predictor-579fd87984-zc225 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a129-predictor-579fd87984-zc225 |
Pulled |
Successfully pulled image "quay.io/opendatahub/error-404-isvc:odh-pr-1928" in 11.594s (11.594s including waiting). Image size: 1355797068 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a129-predictor-5c488f9bbf-ljp4w |
Pulled |
Successfully pulled image "quay.io/opendatahub/success-200-isvc:odh-pr-1928" in 11.382s (11.382s including waiting). Image size: 1356801616 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a129-predictor-5c488f9bbf-ljp4w |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a129-predictor-5c488f9bbf-ljp4w |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a129-predictor-579fd87984-zc225 |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a129-predictor-579fd87984-zc225 |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.24s (2.24s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a129-predictor-579fd87984-zc225 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a129-predictor-579fd87984-zc225 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a129-predictor-5c488f9bbf-ljp4w |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a129-predictor-5c488f9bbf-ljp4w |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.186s (2.186s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a129-predictor-5c488f9bbf-ljp4w |
Started |
Started container kube-rbac-proxy | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4a129-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4a129-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4a129-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4a129-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-ffc456649-z8txx |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-ffc456649-z8txx |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1928" in 27.777s (27.777s including waiting). Image size: 1583203731 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-ffc456649-z8txx |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-ffc456649-z8txx |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xkt8s |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6f7b8d4cb-9t97d |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xkt8s |
Pulled |
Successfully pulled image "kserve/xgbserver:latest" in 28.212s (28.212s including waiting). Image size: 1435742979 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6f7b8d4cb-9t97d |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xkt8s |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xkt8s |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6f7b8d4cb-9t97d |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1928" in 29.332s (29.332s including waiting). Image size: 1583203731 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6f7b8d4cb-9t97d |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xkt8s |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.454s (2.454s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xkt8s |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xkt8s |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-ffc456649-z8txx |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.884s (2.884s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6f7b8d4cb-9t97d |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.333s (2.333s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6f7b8d4cb-9t97d |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6f7b8d4cb-9t97d |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-ffc456649-z8txx |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-ffc456649-z8txx |
Started |
Started container kube-rbac-proxy | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-4a129 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-4a129 |
InferenceServiceReady |
InferenceService [success-200-isvc-4a129] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-4a129 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-4a129 |
InferenceServiceReady |
InferenceService [error-404-isvc-4a129] is Ready | |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4a129-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4a129-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4a129-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4a129-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-4a129 |
ScalingReplicaSet |
Scaled up replica set switch-graph-4a129-8fdcd5874 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-4a129-8fdcd5874 |
SuccessfulCreate |
Created pod: switch-graph-4a129-8fdcd5874-qjxjz | |
kserve-ci-e2e-test |
multus |
switch-graph-4a129-8fdcd5874-qjxjz |
AddedInterface |
Add eth0 [10.134.0.37/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-4a129 |
InferenceGraphReady |
InferenceGraph [switch-graph-4a129] is Ready | |
kserve-ci-e2e-test |
kubelet |
switch-graph-4a129-8fdcd5874-qjxjz |
Pulling |
Pulling image "quay.io/opendatahub/kserve-router@sha256:f6d52988e64cd8a9a21f609c0c944dcdc6613df8ea74ac8dfd85c52003a9bc37" | |
kserve-ci-e2e-test |
kubelet |
switch-graph-4a129-8fdcd5874-qjxjz |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-router@sha256:f6d52988e64cd8a9a21f609c0c944dcdc6613df8ea74ac8dfd85c52003a9bc37" in 1.707s (1.707s including waiting). Image size: 211751512 bytes. | |
kserve-ci-e2e-test |
kubelet |
switch-graph-4a129-8fdcd5874-qjxjz |
Created |
Created container: switch-graph-4a129 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-4a129-8fdcd5874-qjxjz |
Started |
Started container switch-graph-4a129 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-8142b-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-8142b-predictor-7948b5d7c6 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-8142b-predictor-79bffc4977 |
SuccessfulCreate |
Created pod: success-200-isvc-8142b-predictor-79bffc4977-htq8k | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a129-predictor-579fd87984-zc225 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a129-predictor-5c488f9bbf-ljp4w |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-8142b-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-8142b-predictor-79bffc4977 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-8142b-predictor-7948b5d7c6 |
SuccessfulCreate |
Created pod: error-404-isvc-8142b-predictor-7948b5d7c6-zd5ng | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a129-predictor-579fd87984-zc225 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a129-predictor-5c488f9bbf-ljp4w |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
switch-graph-4a129-8fdcd5874-qjxjz |
Killing |
Stopping container switch-graph-4a129 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8142b-predictor-79bffc4977-htq8k |
FailedMount |
MountVolume.SetUp failed for volume "kube-rbac-proxy-sar-config" : failed to sync configmap cache: timed out waiting for the condition | |
kserve-ci-e2e-test |
multus |
success-200-isvc-8142b-predictor-79bffc4977-htq8k |
AddedInterface |
Add eth0 [10.133.0.17/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8142b-predictor-7948b5d7c6-zd5ng |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8142b-predictor-7948b5d7c6-zd5ng |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8142b-predictor-7948b5d7c6-zd5ng |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8142b-predictor-7948b5d7c6-zd5ng |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8142b-predictor-7948b5d7c6-zd5ng |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8142b-predictor-7948b5d7c6-zd5ng |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1928" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-8142b-predictor-7948b5d7c6-zd5ng |
AddedInterface |
Add eth0 [10.134.0.38/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8142b-predictor-79bffc4977-htq8k |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8142b-predictor-79bffc4977-htq8k |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8142b-predictor-79bffc4977-htq8k |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8142b-predictor-79bffc4977-htq8k |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8142b-predictor-79bffc4977-htq8k |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8142b-predictor-79bffc4977-htq8k |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1928" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a129-predictor-5c488f9bbf-ljp4w |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.16:8643/healthz": dial tcp 10.133.0.16:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a129-predictor-579fd87984-zc225 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.36:8643/healthz": dial tcp 10.134.0.36:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a129-predictor-579fd87984-zc225 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.36:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a129-predictor-5c488f9bbf-ljp4w |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.16:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
InferenceServiceReady |
InferenceService [isvc-xgboost-graph] is Ready | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-2] is Ready | |
| (x10) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x8) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-1] is Ready | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-4a129-8fdcd5874-qjxjz |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-8142b-predictor-79bffc4977-htq8k |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.17:8080: connect: connection refused |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8142b-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8142b-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8142b-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8142b-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-8142b |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-8142b |
InferenceServiceReady |
InferenceService [error-404-isvc-8142b] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
model-chainer |
ScalingReplicaSet |
Scaled up replica set model-chainer-597fd9c774 from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-8142b |
InferenceServiceReady |
InferenceService [success-200-isvc-8142b] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-8142b |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
replicaset-controller |
model-chainer-597fd9c774 |
SuccessfulCreate |
Created pod: model-chainer-597fd9c774-rhswc | |
kserve-ci-e2e-test |
kubelet |
model-chainer-597fd9c774-rhswc |
Started |
Started container model-chainer | |
kserve-ci-e2e-test |
kubelet |
model-chainer-597fd9c774-rhswc |
Created |
Created container: model-chainer | |
kserve-ci-e2e-test |
InferenceGraphController |
model-chainer |
InferenceGraphReady |
InferenceGraph [model-chainer] is Ready | |
kserve-ci-e2e-test |
kubelet |
model-chainer-597fd9c774-rhswc |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:f6d52988e64cd8a9a21f609c0c944dcdc6613df8ea74ac8dfd85c52003a9bc37" already present on machine | |
kserve-ci-e2e-test |
multus |
model-chainer-597fd9c774-rhswc |
AddedInterface |
Add eth0 [10.134.0.39/23] from ovn-kubernetes | |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-8142b |
InferenceGraphReady |
InferenceGraph [switch-graph-8142b] is Ready | |
kserve-ci-e2e-test |
kubelet |
model-chainer-597fd9c774-rhswc |
Killing |
Stopping container model-chainer | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-8142b |
ScalingReplicaSet |
Scaled up replica set switch-graph-8142b-d9d5fcc5c from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-8142b-d9d5fcc5c |
SuccessfulCreate |
Created pod: switch-graph-8142b-d9d5fcc5c-79qkv | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6f7b8d4cb-9t97d |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6f7b8d4cb-9t97d |
Killing |
Stopping container kube-rbac-proxy | |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xkt8s |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xkt8s |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-ffc456649-z8txx |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-ffc456649-z8txx |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
switch-graph-8142b-d9d5fcc5c-79qkv |
Started |
Started container switch-graph-8142b | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-dcaf3-predictor-5b5b4dfc-f5n25 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-dcaf3-predictor-5b5b4dfc |
SuccessfulCreate |
Created pod: success-200-isvc-dcaf3-predictor-5b5b4dfc-f5n25 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-dcaf3-predictor-5b5b4dfc-f5n25 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
switch-graph-8142b-d9d5fcc5c-79qkv |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:f6d52988e64cd8a9a21f609c0c944dcdc6613df8ea74ac8dfd85c52003a9bc37" already present on machine | |
kserve-ci-e2e-test |
kubelet |
switch-graph-8142b-d9d5fcc5c-79qkv |
Created |
Created container: switch-graph-8142b | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-dcaf3-predictor-5b5b4dfc-f5n25 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-dcaf3-predictor-5fdbd86d9b |
SuccessfulCreate |
Created pod: error-404-isvc-dcaf3-predictor-5fdbd86d9b-xb9bl | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-dcaf3-predictor-5b5b4dfc-f5n25 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-dcaf3-predictor-5b5b4dfc-f5n25 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1928" already present on machine | |
kserve-ci-e2e-test |
multus |
switch-graph-8142b-d9d5fcc5c-79qkv |
AddedInterface |
Add eth0 [10.134.0.40/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-dcaf3-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-dcaf3-predictor-5b5b4dfc from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-dcaf3-predictor-5b5b4dfc-f5n25 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
success-200-isvc-dcaf3-predictor-5b5b4dfc-f5n25 |
AddedInterface |
Add eth0 [10.133.0.18/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-dcaf3-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-dcaf3-predictor-5fdbd86d9b from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-dcaf3-predictor-5fdbd86d9b-xb9bl |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-dcaf3-predictor-5fdbd86d9b-xb9bl |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-dcaf3-predictor-5fdbd86d9b-xb9bl |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-dcaf3-predictor-5fdbd86d9b-xb9bl |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-dcaf3-predictor-5fdbd86d9b-xb9bl |
AddedInterface |
Add eth0 [10.134.0.41/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-dcaf3-predictor-5fdbd86d9b-xb9bl |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1928" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-dcaf3-predictor-5fdbd86d9b-xb9bl |
Created |
Created container: kserve-container | |
| (x10) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-ffc456649-z8txx |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.25:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-ffc456649-z8txx |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.25:8643/healthz": dial tcp 10.132.0.25:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xkt8s |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.24:8643/healthz": dial tcp 10.132.0.24:8643: connect: connection refused | |
| (x9) | kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xkt8s |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.24:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6f7b8d4cb-9t97d |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.23:8643/healthz": dial tcp 10.132.0.23:8643: connect: connection refused | |
| (x10) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-6f7b8d4cb-9t97d |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.23:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8142b-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8142b-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-dcaf3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-dcaf3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
model-chainer-597fd9c774-rhswc |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-dcaf3-predictor-5b5b4dfc-f5n25 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.18:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8142b-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-8142b |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-8142b |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8142b-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-dcaf3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-dcaf3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-dcaf3 |
InferenceServiceReady |
InferenceService [error-404-isvc-dcaf3] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-dcaf3 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-dcaf3 |
InferenceServiceReady |
InferenceService [success-200-isvc-dcaf3] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-dcaf3 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-dcaf3 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-dcaf3-8fd969546 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-dcaf3-8fd969546 |
SuccessfulCreate |
Created pod: sequence-graph-dcaf3-8fd969546-vzrkj | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-dcaf3-8fd969546-vzrkj |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:f6d52988e64cd8a9a21f609c0c944dcdc6613df8ea74ac8dfd85c52003a9bc37" already present on machine | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-dcaf3-8fd969546-vzrkj |
Started |
Started container sequence-graph-dcaf3 | |
kserve-ci-e2e-test |
multus |
sequence-graph-dcaf3-8fd969546-vzrkj |
AddedInterface |
Add eth0 [10.134.0.42/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-dcaf3-8fd969546-vzrkj |
Created |
Created container: sequence-graph-dcaf3 | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-dcaf3 |
InferenceGraphReady |
InferenceGraph [sequence-graph-dcaf3] is Ready | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-dcaf3 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-dcaf3 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-dcaf3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-dcaf3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-dcaf3 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-dcaf3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-dcaf3 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-dcaf3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
switch-graph-8142b-d9d5fcc5c-79qkv |
Killing |
Stopping container switch-graph-8142b | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1fda1-predictor-67c979686f-cnskz |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1928" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8142b-predictor-7948b5d7c6-zd5ng |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1fda1-predictor-67c979686f-cnskz |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8142b-predictor-79bffc4977-htq8k |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8142b-predictor-79bffc4977-htq8k |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-1fda1-predictor-67c979686f |
SuccessfulCreate |
Created pod: success-200-isvc-1fda1-predictor-67c979686f-cnskz | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-1fda1 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-1fda1": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-1fda1 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-1fda1": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-1fda1": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-1fda1-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-1fda1-predictor-67c979686f from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1fda1-predictor-c86ccb888-n4lkk |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-1fda1-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1fda1-predictor-67c979686f-cnskz |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1fda1-predictor-67c979686f-cnskz |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-1fda1-predictor-c86ccb888 |
SuccessfulCreate |
Created pod: error-404-isvc-1fda1-predictor-c86ccb888-n4lkk | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-1fda1-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-1fda1-predictor-c86ccb888 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8142b-predictor-7948b5d7c6-zd5ng |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-1fda1 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-1fda1": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-1fda1 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-1fda1": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-1fda1": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
multus |
success-200-isvc-1fda1-predictor-67c979686f-cnskz |
AddedInterface |
Add eth0 [10.133.0.19/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1fda1-predictor-c86ccb888-n4lkk |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1928" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1fda1-predictor-c86ccb888-n4lkk |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1fda1-predictor-67c979686f-cnskz |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1fda1-predictor-67c979686f-cnskz |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1fda1-predictor-c86ccb888-n4lkk |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1fda1-predictor-c86ccb888-n4lkk |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1fda1-predictor-c86ccb888-n4lkk |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1fda1-predictor-c86ccb888-n4lkk |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-1fda1-predictor-c86ccb888-n4lkk |
AddedInterface |
Add eth0 [10.134.0.43/23] from ovn-kubernetes | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-8142b-predictor-7948b5d7c6-zd5ng |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.38:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8142b-predictor-7948b5d7c6-zd5ng |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.38:8643/healthz": dial tcp 10.134.0.38:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-8142b-d9d5fcc5c-79qkv |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-1fda1-predictor-67c979686f-cnskz |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.19:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-1fda1-predictor-c86ccb888-n4lkk |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.43:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1fda1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1fda1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-1fda1 |
InferenceServiceReady |
InferenceService [success-200-isvc-1fda1] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-1fda1 |
InferenceServiceReady |
InferenceService [error-404-isvc-1fda1] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-1fda1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-1fda1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1fda1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
sequence-graph-dcaf3-8fd969546-vzrkj |
Killing |
Stopping container sequence-graph-dcaf3 | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-1fda1 |
UpdateFailed |
Failed to update status for InferenceGraph "ensemble-graph-1fda1": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "ensemble-graph-1fda1": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-1fda1 |
InferenceGraphReady |
InferenceGraph [ensemble-graph-1fda1] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-1fda1 |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-1fda1-756f77b956 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-c397f-predictor-84cc68fd6c |
SuccessfulCreate |
Created pod: success-200-isvc-c397f-predictor-84cc68fd6c-lzhj9 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-dcaf3-predictor-5b5b4dfc-f5n25 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-dcaf3-predictor-5fdbd86d9b-xb9bl |
Killing |
Stopping container kube-rbac-proxy | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1fda1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-1fda1-756f77b956 |
SuccessfulCreate |
Created pod: ensemble-graph-1fda1-756f77b956-96xnd | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1fda1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-dcaf3-predictor-5fdbd86d9b-xb9bl |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-dcaf3-predictor-5b5b4dfc-f5n25 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-1fda1-756f77b956-96xnd |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:f6d52988e64cd8a9a21f609c0c944dcdc6613df8ea74ac8dfd85c52003a9bc37" already present on machine | |
kserve-ci-e2e-test |
multus |
ensemble-graph-1fda1-756f77b956-96xnd |
AddedInterface |
Add eth0 [10.134.0.44/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1fda1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-1fda1 |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "ensemble-graph-1fda1": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-c397f-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-c397f-predictor-84cc68fd6c from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c397f-predictor-699fb4686-ws4j5 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c397f-predictor-84cc68fd6c-lzhj9 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-c397f-predictor-84cc68fd6c-lzhj9 |
AddedInterface |
Add eth0 [10.133.0.20/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c397f-predictor-84cc68fd6c-lzhj9 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1928" already present on machine | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-1fda1-756f77b956-96xnd |
Started |
Started container ensemble-graph-1fda1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c397f-predictor-699fb4686-ws4j5 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-c397f-predictor-699fb4686 |
SuccessfulCreate |
Created pod: error-404-isvc-c397f-predictor-699fb4686-ws4j5 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c397f-predictor-84cc68fd6c-lzhj9 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-c397f-predictor-699fb4686-ws4j5 |
AddedInterface |
Add eth0 [10.134.0.45/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c397f-predictor-699fb4686-ws4j5 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1928" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c397f-predictor-699fb4686-ws4j5 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c397f-predictor-699fb4686-ws4j5 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c397f-predictor-84cc68fd6c-lzhj9 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-c397f-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-c397f-predictor-699fb4686 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c397f-predictor-699fb4686-ws4j5 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-1fda1-756f77b956-96xnd |
Created |
Created container: ensemble-graph-1fda1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c397f-predictor-84cc68fd6c-lzhj9 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c397f-predictor-84cc68fd6c-lzhj9 |
Started |
Started container kube-rbac-proxy | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-dcaf3-predictor-5fdbd86d9b-xb9bl |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.41:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-dcaf3-predictor-5fdbd86d9b-xb9bl |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.41:8643/healthz": dial tcp 10.134.0.41:8643: connect: connection refused | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-1175d-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-1175d-predictor-c88c69555 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-1175d-predictor-c88c69555 |
SuccessfulCreate |
Created pod: success-200-isvc-1175d-predictor-c88c69555-c5nkn | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-1fda1-756f77b956-96xnd |
Killing |
Stopping container ensemble-graph-1fda1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1fda1-predictor-67c979686f-cnskz |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1fda1-predictor-c86ccb888-n4lkk |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1fda1-predictor-c86ccb888-n4lkk |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1fda1-predictor-67c979686f-cnskz |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-1175d-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-1175d-predictor-6784b69858 from 0 to 1 | |
| (x2) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-1175d-predictor-c88c69555-c5nkn |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-1175d-predictor-serving-cert" not found |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1175d-predictor-6784b69858-d8x9r |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1175d-predictor-6784b69858-d8x9r |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1928" already present on machine | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-1175d-predictor-6784b69858 |
SuccessfulCreate |
Created pod: error-404-isvc-1175d-predictor-6784b69858-d8x9r | |
kserve-ci-e2e-test |
multus |
error-404-isvc-1175d-predictor-6784b69858-d8x9r |
AddedInterface |
Add eth0 [10.134.0.46/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1175d-predictor-6784b69858-d8x9r |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1175d-predictor-6784b69858-d8x9r |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1175d-predictor-6784b69858-d8x9r |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1175d-predictor-6784b69858-d8x9r |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c397f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c397f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
multus |
success-200-isvc-1175d-predictor-c88c69555-c5nkn |
AddedInterface |
Add eth0 [10.133.0.21/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1175d-predictor-c88c69555-c5nkn |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1175d-predictor-c88c69555-c5nkn |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1175d-predictor-c88c69555-c5nkn |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1175d-predictor-c88c69555-c5nkn |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1928" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1175d-predictor-c88c69555-c5nkn |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1175d-predictor-c88c69555-c5nkn |
Started |
Started container kube-rbac-proxy | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-dcaf3-8fd969546-vzrkj |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-c397f-predictor-84cc68fd6c-lzhj9 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.20:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-c397f-predictor-699fb4686-ws4j5 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.45:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-1fda1-756f77b956-96xnd |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c397f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c397f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-c397f |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-c397f |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-c397f |
InferenceServiceReady |
InferenceService [success-200-isvc-c397f] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-c397f |
InferenceServiceReady |
InferenceService [error-404-isvc-c397f] is Ready | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-1175d-predictor-c88c69555-c5nkn |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.21:8080: connect: connection refused |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1175d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1175d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-c397f |
ScalingReplicaSet |
Scaled up replica set sequence-graph-c397f-66d6dcb945 from 0 to 1 | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1175d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1175d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c397f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c397f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-c397f-66d6dcb945 |
SuccessfulCreate |
Created pod: sequence-graph-c397f-66d6dcb945-6dwwc | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-c397f |
InferenceGraphReady |
InferenceGraph [sequence-graph-c397f] is Ready | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-c397f-66d6dcb945-6dwwc |
Started |
Started container sequence-graph-c397f | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-c397f-66d6dcb945-6dwwc |
Created |
Created container: sequence-graph-c397f | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-c397f-66d6dcb945-6dwwc |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:f6d52988e64cd8a9a21f609c0c944dcdc6613df8ea74ac8dfd85c52003a9bc37" already present on machine | |
kserve-ci-e2e-test |
multus |
sequence-graph-c397f-66d6dcb945-6dwwc |
AddedInterface |
Add eth0 [10.134.0.47/23] from ovn-kubernetes | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c397f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c397f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-1175d |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-1175d |
InferenceServiceReady |
InferenceService [error-404-isvc-1175d] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-1175d |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-1175d |
InferenceServiceReady |
InferenceService [success-200-isvc-1175d] is Ready | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c397f-predictor-84cc68fd6c-lzhj9 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c397f-predictor-84cc68fd6c-lzhj9 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-c397f-66d6dcb945-6dwwc |
Killing |
Stopping container sequence-graph-c397f | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1e4e7-predictor-76564f499-76j8q |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1e4e7-predictor-76564f499-76j8q |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-1e4e7-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-1e4e7-predictor-86f4b8b95d from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c397f-predictor-699fb4686-ws4j5 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-1e4e7-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-1e4e7-predictor-76564f499 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-1e4e7-predictor-76564f499 |
SuccessfulCreate |
Created pod: success-200-isvc-1e4e7-predictor-76564f499-76j8q | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-1e4e7-predictor-86f4b8b95d |
SuccessfulCreate |
Created pod: error-404-isvc-1e4e7-predictor-86f4b8b95d-glqzn | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1e4e7-predictor-76564f499-76j8q |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1e4e7-predictor-86f4b8b95d-glqzn |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-1e4e7-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1e4e7-predictor-76564f499-76j8q |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c397f-predictor-699fb4686-ws4j5 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1e4e7-predictor-76564f499-76j8q |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
success-200-isvc-1e4e7-predictor-76564f499-76j8q |
AddedInterface |
Add eth0 [10.133.0.22/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1e4e7-predictor-76564f499-76j8q |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1928" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1e4e7-predictor-86f4b8b95d-glqzn |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1e4e7-predictor-86f4b8b95d-glqzn |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1e4e7-predictor-86f4b8b95d-glqzn |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-1e4e7-predictor-86f4b8b95d-glqzn |
AddedInterface |
Add eth0 [10.134.0.48/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1e4e7-predictor-86f4b8b95d-glqzn |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1928" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1e4e7-predictor-86f4b8b95d-glqzn |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1e4e7-predictor-86f4b8b95d-glqzn |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c397f-predictor-699fb4686-ws4j5 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.45:8643/healthz": dial tcp 10.134.0.45:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c397f-predictor-84cc68fd6c-lzhj9 |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.20:8643/healthz": dial tcp 10.133.0.20:8643: connect: connection refused | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-1175d |
InferenceGraphReady |
InferenceGraph [ensemble-graph-1175d] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-1175d |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-1175d-68b6c76bfd from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-1175d-68b6c76bfd |
SuccessfulCreate |
Created pod: ensemble-graph-1175d-68b6c76bfd-vkdb6 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-1175d-68b6c76bfd-vkdb6 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "ensemble-graph-1175d-serving-cert" not found | |
kserve-ci-e2e-test |
multus |
ensemble-graph-1175d-68b6c76bfd-vkdb6 |
AddedInterface |
Add eth0 [10.134.0.49/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-1175d-68b6c76bfd-vkdb6 |
Started |
Started container ensemble-graph-1175d | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-1175d-68b6c76bfd-vkdb6 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:f6d52988e64cd8a9a21f609c0c944dcdc6613df8ea74ac8dfd85c52003a9bc37" already present on machine | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-1175d-68b6c76bfd-vkdb6 |
Created |
Created container: ensemble-graph-1175d | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1175d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1175d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1175d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1175d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-c397f-66d6dcb945-6dwwc |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-1175d |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-1175d |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1e4e7-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1e4e7-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-1175d |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-1175d |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1e4e7-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1e4e7-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-1e4e7 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-1e4e7 |
InferenceServiceReady |
InferenceService [error-404-isvc-1e4e7] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-1e4e7 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-1e4e7 |
InferenceServiceReady |
InferenceService [success-200-isvc-1e4e7] is Ready | |
kserve-ci-e2e-test |
multus |
sequence-graph-1e4e7-6f5ffcc466-vpjqm |
AddedInterface |
Add eth0 [10.134.0.50/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-1e4e7-6f5ffcc466 |
SuccessfulCreate |
Created pod: sequence-graph-1e4e7-6f5ffcc466-vpjqm | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-1e4e7-6f5ffcc466-vpjqm |
Started |
Started container sequence-graph-1e4e7 | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-1e4e7 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-1e4e7-6f5ffcc466 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-1e4e7 |
InferenceGraphReady |
InferenceGraph [sequence-graph-1e4e7] is Ready | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-1e4e7-6f5ffcc466-vpjqm |
Created |
Created container: sequence-graph-1e4e7 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-1e4e7-6f5ffcc466-vpjqm |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:f6d52988e64cd8a9a21f609c0c944dcdc6613df8ea74ac8dfd85c52003a9bc37" already present on machine | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-1e4e7 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1e4e7-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1e4e7-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-1e4e7 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1e4e7-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-1e4e7 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-1e4e7 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1e4e7-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-9819a-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-9819a-predictor-75b9f568db from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-9819a-predictor-75b9f568db |
SuccessfulCreate |
Created pod: error-404-isvc-9819a-predictor-75b9f568db-gpdmf | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-9819a-predictor-69df5dcdcf |
SuccessfulCreate |
Created pod: success-200-isvc-9819a-predictor-69df5dcdcf-bn5vz | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1175d-predictor-6784b69858-d8x9r |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1175d-predictor-6784b69858-d8x9r |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1175d-predictor-c88c69555-c5nkn |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1175d-predictor-c88c69555-c5nkn |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-1175d-68b6c76bfd-vkdb6 |
Killing |
Stopping container ensemble-graph-1175d | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-9819a-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-9819a-predictor-69df5dcdcf from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1175d-predictor-c88c69555-c5nkn |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.21:8643/healthz": dial tcp 10.133.0.21:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9819a-predictor-75b9f568db-gpdmf |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-9819a-predictor-serving-cert" not found | |
| (x2) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-9819a-predictor-69df5dcdcf-bn5vz |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-9819a-predictor-serving-cert" not found |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9819a-predictor-75b9f568db-gpdmf |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9819a-predictor-75b9f568db-gpdmf |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9819a-predictor-69df5dcdcf-bn5vz |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1928" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9819a-predictor-69df5dcdcf-bn5vz |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9819a-predictor-69df5dcdcf-bn5vz |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9819a-predictor-69df5dcdcf-bn5vz |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9819a-predictor-69df5dcdcf-bn5vz |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9819a-predictor-69df5dcdcf-bn5vz |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9819a-predictor-75b9f568db-gpdmf |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9819a-predictor-75b9f568db-gpdmf |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
success-200-isvc-9819a-predictor-69df5dcdcf-bn5vz |
AddedInterface |
Add eth0 [10.133.0.23/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9819a-predictor-75b9f568db-gpdmf |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-9819a-predictor-75b9f568db-gpdmf |
AddedInterface |
Add eth0 [10.134.0.51/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9819a-predictor-75b9f568db-gpdmf |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1928" already present on machine | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-1175d-predictor-6784b69858-d8x9r |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.46:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1175d-predictor-6784b69858-d8x9r |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.46:8643/healthz": dial tcp 10.134.0.46:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-1175d-68b6c76bfd-vkdb6 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-9819a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-9819a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-9819a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-9819a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-9819a-predictor-75b9f568db-gpdmf |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.51:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-9819a-predictor-69df5dcdcf-bn5vz |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.23:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1e4e7-predictor-76564f499-76j8q |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1e4e7-predictor-76564f499-76j8q |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-5ec4c-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-5ec4c-predictor-5d6f7b484c from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-1e4e7-6f5ffcc466-vpjqm |
Killing |
Stopping container sequence-graph-1e4e7 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5ec4c-predictor-68b6c44d66-hmwnv |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-5ec4c-predictor-68b6c44d66 |
SuccessfulCreate |
Created pod: error-404-isvc-5ec4c-predictor-68b6c44d66-hmwnv | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1e4e7-predictor-86f4b8b95d-glqzn |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-5ec4c-predictor-68b6c44d66-hmwnv |
AddedInterface |
Add eth0 [10.134.0.52/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5ec4c-predictor-5d6f7b484c-9z4xt |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1928" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5ec4c-predictor-68b6c44d66-hmwnv |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1928" already present on machine | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-5ec4c |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-5ec4c": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-5ec4c |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-5ec4c": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-5ec4c": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5ec4c-predictor-68b6c44d66-hmwnv |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5ec4c-predictor-68b6c44d66-hmwnv |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5ec4c-predictor-68b6c44d66-hmwnv |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5ec4c-predictor-5d6f7b484c-9z4xt |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5ec4c-predictor-5d6f7b484c-9z4xt |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5ec4c-predictor-5d6f7b484c-9z4xt |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1e4e7-predictor-86f4b8b95d-glqzn |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5ec4c-predictor-5d6f7b484c-9z4xt |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5ec4c-predictor-68b6c44d66-hmwnv |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5ec4c-predictor-5d6f7b484c-9z4xt |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-5ec4c-predictor-5d6f7b484c |
SuccessfulCreate |
Created pod: success-200-isvc-5ec4c-predictor-5d6f7b484c-9z4xt | |
kserve-ci-e2e-test |
multus |
success-200-isvc-5ec4c-predictor-5d6f7b484c-9z4xt |
AddedInterface |
Add eth0 [10.133.0.24/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-5ec4c-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-5ec4c-predictor-68b6c44d66 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1e4e7-predictor-76564f499-76j8q |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.22:8643/healthz": dial tcp 10.133.0.22:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1e4e7-predictor-86f4b8b95d-glqzn |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.48:8643/healthz": dial tcp 10.134.0.48:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-1e4e7-predictor-76564f499-76j8q |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.22:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-1e4e7-predictor-86f4b8b95d-glqzn |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.48:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-9819a |
InferenceServiceReady |
InferenceService [success-200-isvc-9819a] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-9819a |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-9819a |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-9819a |
InferenceServiceReady |
InferenceService [error-404-isvc-9819a] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-9819a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-9819a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-9819a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-9819a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-9819a |
ScalingReplicaSet |
Scaled up replica set splitter-graph-9819a-5b6b565754 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-9819a-5b6b565754 |
SuccessfulCreate |
Created pod: splitter-graph-9819a-5b6b565754-mn9bp | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-9819a-5b6b565754-mn9bp |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "splitter-graph-9819a-serving-cert" not found | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-9819a |
InferenceGraphReady |
InferenceGraph [splitter-graph-9819a] is Ready | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-9819a-5b6b565754-mn9bp |
Started |
Started container splitter-graph-9819a | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-9819a-5b6b565754-mn9bp |
Created |
Created container: splitter-graph-9819a | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-9819a-5b6b565754-mn9bp |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:f6d52988e64cd8a9a21f609c0c944dcdc6613df8ea74ac8dfd85c52003a9bc37" already present on machine | |
kserve-ci-e2e-test |
multus |
splitter-graph-9819a-5b6b565754-mn9bp |
AddedInterface |
Add eth0 [10.134.0.53/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9819a-predictor-75b9f568db-gpdmf |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9819a-predictor-69df5dcdcf-bn5vz |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-9819a-predictor-69df5dcdcf-bn5vz |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-9819a-predictor-75b9f568db-gpdmf |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-9819a-5b6b565754-mn9bp |
Killing |
Stopping container splitter-graph-9819a | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-892d5-predictor-6659f5749d |
SuccessfulCreate |
Created pod: error-404-isvc-892d5-predictor-6659f5749d-6qmvb | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-892d5-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-892d5-predictor-6659f5749d from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-892d5-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-892d5-predictor-b75d9549f from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-892d5-predictor-b75d9549f |
SuccessfulCreate |
Created pod: success-200-isvc-892d5-predictor-b75d9549f-ffcm2 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-892d5-predictor-6659f5749d-6qmvb |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-892d5-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-892d5-predictor-b75d9549f-ffcm2 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : failed to sync secret cache: timed out waiting for the condition | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-892d5-predictor-6659f5749d-6qmvb |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1928" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-892d5-predictor-6659f5749d-6qmvb |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-892d5-predictor-6659f5749d-6qmvb |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-892d5-predictor-6659f5749d-6qmvb |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-892d5-predictor-6659f5749d-6qmvb |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-892d5-predictor-6659f5749d-6qmvb |
AddedInterface |
Add eth0 [10.134.0.54/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-892d5-predictor-6659f5749d-6qmvb |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-892d5-predictor-b75d9549f-ffcm2 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1928" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-892d5-predictor-b75d9549f-ffcm2 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-892d5-predictor-b75d9549f-ffcm2 |
AddedInterface |
Add eth0 [10.133.0.25/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-892d5-predictor-b75d9549f-ffcm2 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-892d5-predictor-b75d9549f-ffcm2 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-892d5-predictor-b75d9549f-ffcm2 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-892d5-predictor-b75d9549f-ffcm2 |
Created |
Created container: kserve-container | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-1e4e7-6f5ffcc466-vpjqm |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-5ec4c-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-5ec4c-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-5ec4c-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-5ec4c-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-5ec4c-predictor-68b6c44d66-hmwnv |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.52:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-5ec4c-predictor-5d6f7b484c-9z4xt |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.24:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-5ec4c |
InferenceServiceReady |
InferenceService [error-404-isvc-5ec4c] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-5ec4c |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-5ec4c |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-5ec4c |
InferenceServiceReady |
InferenceService [success-200-isvc-5ec4c] is Ready | |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-9819a-5b6b565754-mn9bp |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-892d5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-892d5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-5ec4c-58bffd5c64 |
SuccessfulCreate |
Created pod: switch-graph-5ec4c-58bffd5c64-54zd4 | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-5ec4c |
InferenceGraphReady |
InferenceGraph [switch-graph-5ec4c] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-5ec4c |
ScalingReplicaSet |
Scaled up replica set switch-graph-5ec4c-58bffd5c64 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-5ec4c-58bffd5c64-54zd4 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-5ec4c-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
switch-graph-5ec4c-58bffd5c64-54zd4 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:f6d52988e64cd8a9a21f609c0c944dcdc6613df8ea74ac8dfd85c52003a9bc37" already present on machine | |
kserve-ci-e2e-test |
kubelet |
switch-graph-5ec4c-58bffd5c64-54zd4 |
Created |
Created container: switch-graph-5ec4c | |
kserve-ci-e2e-test |
kubelet |
switch-graph-5ec4c-58bffd5c64-54zd4 |
Started |
Started container switch-graph-5ec4c | |
kserve-ci-e2e-test |
multus |
switch-graph-5ec4c-58bffd5c64-54zd4 |
AddedInterface |
Add eth0 [10.134.0.55/23] from ovn-kubernetes | |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-892d5-predictor-6659f5749d-6qmvb |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.54:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-892d5-predictor-b75d9549f-ffcm2 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.25:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-892d5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-892d5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-892d5 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-892d5 |
InferenceServiceReady |
InferenceService [error-404-isvc-892d5] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-892d5 |
InferenceServiceReady |
InferenceService [success-200-isvc-892d5] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-892d5 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-5ec4c-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-5ec4c-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-892d5 |
ScalingReplicaSet |
Scaled up replica set splitter-graph-892d5-75f7f9d99c from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-892d5 |
InferenceGraphReady |
InferenceGraph [splitter-graph-892d5] is Ready | |
kserve-ci-e2e-test |
multus |
splitter-graph-892d5-75f7f9d99c-4gtmq |
AddedInterface |
Add eth0 [10.134.0.56/23] from ovn-kubernetes | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-892d5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-892d5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
splitter-graph-892d5-75f7f9d99c-4gtmq |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:f6d52988e64cd8a9a21f609c0c944dcdc6613df8ea74ac8dfd85c52003a9bc37" already present on machine | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-892d5-75f7f9d99c-4gtmq |
Created |
Created container: splitter-graph-892d5 | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-892d5-75f7f9d99c-4gtmq |
Started |
Started container splitter-graph-892d5 | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-892d5-75f7f9d99c |
SuccessfulCreate |
Created pod: splitter-graph-892d5-75f7f9d99c-4gtmq | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-5ec4c |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-5ec4c |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-892d5 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-892d5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-892d5 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-892d5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-5ec4c-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-5ec4c-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-892d5-predictor-6659f5749d-6qmvb |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-892d5-predictor-6659f5749d-6qmvb |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-892d5-75f7f9d99c-4gtmq |
Killing |
Stopping container splitter-graph-892d5 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-892d5-predictor-b75d9549f-ffcm2 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-892d5-predictor-b75d9549f-ffcm2 |
Killing |
Stopping container kserve-container | |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-892d5-75f7f9d99c-4gtmq |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
switch-graph-5ec4c-58bffd5c64-54zd4 |
Killing |
Stopping container switch-graph-5ec4c | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5ec4c-predictor-5d6f7b484c-9z4xt |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5ec4c-predictor-68b6c44d66-hmwnv |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5ec4c-predictor-68b6c44d66-hmwnv |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5ec4c-predictor-5d6f7b484c-9z4xt |
Killing |
Stopping container kserve-container | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-5ec4c-58bffd5c64-54zd4 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |