| Time | Namespace | Component | RelatedObject | Reason | Message |
|---|---|---|---|---|---|
kserve-ci-e2e-test |
success-200-isvc-53cb1-predictor-77bd468459-7gnsg |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-53cb1-predictor-77bd468459-7gnsg to ip-10-0-137-234.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-81279-predictor-84575cb8cb-pllk2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-81279-predictor-84575cb8cb-pllk2 to ip-10-0-128-184.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-90077-predictor-7767f7d476-wsrwq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-90077-predictor-7767f7d476-wsrwq to ip-10-0-128-184.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-510d5-predictor-8556d94969-wh62f |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-510d5-predictor-8556d94969-wh62f to ip-10-0-137-234.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-510d5-predictor-5ccb9788d8-cqw5l |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-510d5-predictor-5ccb9788d8-cqw5l to ip-10-0-128-184.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-1-predictor-869f8dd9fc-vpddx |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-869f8dd9fc-vpddx to ip-10-0-142-76.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-2e273-predictor-6db6bf4f6-6q6b2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-2e273-predictor-6db6bf4f6-6q6b2 to ip-10-0-128-184.ec2.internal | ||
kserve-ci-e2e-test |
isvc-xgboost-graph-predictor-555fc69d56-ksclf |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ksclf to ip-10-0-137-234.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-7dad6-predictor-6c8d884d74-7hlfm |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-7dad6-predictor-6c8d884d74-7hlfm to ip-10-0-128-184.ec2.internal | ||
kserve-ci-e2e-test |
model-chainer-6f496df4c6-8w6hl |
Scheduled |
Successfully assigned kserve-ci-e2e-test/model-chainer-6f496df4c6-8w6hl to ip-10-0-137-234.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-326f5-predictor-669c54dd8c-85sxw |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-326f5-predictor-669c54dd8c-85sxw to ip-10-0-137-234.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-6a9b1-predictor-8496c76f6d-6c6kk |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-6a9b1-predictor-8496c76f6d-6c6kk to ip-10-0-128-184.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-510d5-85cb7dd998-8x2xs |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-510d5-85cb7dd998-8x2xs to ip-10-0-137-234.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-2e273-55d76c96b5-pbttp |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-2e273-55d76c96b5-pbttp to ip-10-0-137-234.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-b3d7b-predictor-5b7c8bf675-xwlk6 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-b3d7b-predictor-5b7c8bf675-xwlk6 to ip-10-0-137-234.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-b3d7b-6c989bffcf-kgvbk |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-b3d7b-6c989bffcf-kgvbk to ip-10-0-137-234.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-90077-predictor-58445bbccc-n7mjk |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-90077-predictor-58445bbccc-n7mjk to ip-10-0-137-234.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-81279-predictor-bb8f78f5c-qlsqq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-81279-predictor-bb8f78f5c-qlsqq to ip-10-0-137-234.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-7dad6-predictor-7c8fc894cf-46kb9 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-7dad6-predictor-7c8fc894cf-46kb9 to ip-10-0-137-234.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-6a9b1-predictor-7bc68bf869-7z2fw |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-6a9b1-predictor-7bc68bf869-7z2fw to ip-10-0-137-234.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-28e88-predictor-6bb6f499b4-jg8ns |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-28e88-predictor-6bb6f499b4-jg8ns to ip-10-0-128-184.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-28e88-f549854fc-7qtn8 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-28e88-f549854fc-7qtn8 to ip-10-0-137-234.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-81279-7ff94b5fb-vhq95 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-81279-7ff94b5fb-vhq95 to ip-10-0-137-234.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-53cb1-d889798ff-6bkzb |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-53cb1-d889798ff-6bkzb to ip-10-0-137-234.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-6a9b1-6d87596d7f-pdg8z |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-6a9b1-6d87596d7f-pdg8z to ip-10-0-137-234.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-326f5-predictor-5f77687698-df8ms |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-326f5-predictor-5f77687698-df8ms to ip-10-0-128-184.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-b3d7b-predictor-544b6dfd74-mbzlt |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-b3d7b-predictor-544b6dfd74-mbzlt to ip-10-0-128-184.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-326f5-768586467d-gw8kd |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-326f5-768586467d-gw8kd to ip-10-0-137-234.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-2-predictor-5679f69dd7-ktwmr |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5679f69dd7-ktwmr to ip-10-0-142-76.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-7dad6-75dfbcc7d7-nd9vt |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-7dad6-75dfbcc7d7-nd9vt to ip-10-0-137-234.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-53cb1-predictor-7578994fd6-m8xvq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-53cb1-predictor-7578994fd6-m8xvq to ip-10-0-128-184.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-90077-7b4844c498-zld4d |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-90077-7b4844c498-zld4d to ip-10-0-137-234.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-2e273-predictor-85c84c9849-pcfd9 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-2e273-predictor-85c84c9849-pcfd9 to ip-10-0-137-234.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-28e88-predictor-779f848669-jfwgg |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-28e88-predictor-779f848669-jfwgg to ip-10-0-137-234.ec2.internal | ||
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-510d5-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-510d5-predictor-8556d94969 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-2-predictor-5679f69dd7 |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-2-predictor-5679f69dd7-ktwmr | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-1-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-1-predictor-869f8dd9fc from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-510d5-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-510d5-predictor-5ccb9788d8 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-1-predictor-869f8dd9fc |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-1-predictor-869f8dd9fc-vpddx | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-510d5-predictor-8556d94969 |
SuccessfulCreate |
Created pod: success-200-isvc-510d5-predictor-8556d94969-wh62f | |
kserve-ci-e2e-test |
deployment-controller |
isvc-xgboost-graph-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-xgboost-graph-predictor-555fc69d56 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-869f8dd9fc-vpddx |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "isvc-sklearn-graph-1-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-510d5-predictor-5ccb9788d8 |
SuccessfulCreate |
Created pod: error-404-isvc-510d5-predictor-5ccb9788d8-cqw5l | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-2-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-2-predictor-5679f69dd7 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-510d5-predictor-8556d94969-wh62f |
Pulling |
Pulling image "quay.io/opendatahub/success-200-isvc:odh-pr-1968" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-ksclf |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "isvc-xgboost-graph-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-510d5-predictor-5ccb9788d8-cqw5l |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-510d5-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
multus |
success-200-isvc-510d5-predictor-8556d94969-wh62f |
AddedInterface |
Add eth0 [10.133.0.26/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-xgboost-graph-predictor-555fc69d56 |
SuccessfulCreate |
Created pod: isvc-xgboost-graph-predictor-555fc69d56-ksclf | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-869f8dd9fc-vpddx |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-1-predictor-869f8dd9fc-vpddx |
AddedInterface |
Add eth0 [10.134.0.17/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-510d5-predictor-5ccb9788d8-cqw5l |
Pulling |
Pulling image "quay.io/opendatahub/error-404-isvc:odh-pr-1968" | |
kserve-ci-e2e-test |
multus |
error-404-isvc-510d5-predictor-5ccb9788d8-cqw5l |
AddedInterface |
Add eth0 [10.132.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-869f8dd9fc-vpddx |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-ksclf |
Pulling |
Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:7707b8c08d87863b7d6818229e866c981f3cd2a84b4ac5b0b7093566775f9a74" | |
kserve-ci-e2e-test |
multus |
isvc-xgboost-graph-predictor-555fc69d56-ksclf |
AddedInterface |
Add eth0 [10.133.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-869f8dd9fc-vpddx |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7707b8c08d87863b7d6818229e866c981f3cd2a84b4ac5b0b7093566775f9a74" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5679f69dd7-ktwmr |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5679f69dd7-ktwmr |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5679f69dd7-ktwmr |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7707b8c08d87863b7d6818229e866c981f3cd2a84b4ac5b0b7093566775f9a74" already present on machine | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-2-predictor-5679f69dd7-ktwmr |
AddedInterface |
Add eth0 [10.134.0.18/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5679f69dd7-ktwmr |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1968" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-869f8dd9fc-vpddx |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1968" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-510d5-predictor-5ccb9788d8-cqw5l |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-510d5-predictor-5ccb9788d8-cqw5l |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-510d5-predictor-5ccb9788d8-cqw5l |
Pulled |
Successfully pulled image "quay.io/opendatahub/error-404-isvc:odh-pr-1968" in 11.988s (11.988s including waiting). Image size: 1355752521 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-510d5-predictor-5ccb9788d8-cqw5l |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-510d5-predictor-8556d94969-wh62f |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-ksclf |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-510d5-predictor-8556d94969-wh62f |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-ksclf |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-510d5-predictor-8556d94969-wh62f |
Pulled |
Successfully pulled image "quay.io/opendatahub/success-200-isvc:odh-pr-1968" in 13.871s (13.871s including waiting). Image size: 1356716620 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-510d5-predictor-8556d94969-wh62f |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-ksclf |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:7707b8c08d87863b7d6818229e866c981f3cd2a84b4ac5b0b7093566775f9a74" in 12.649s (12.649s including waiting). Image size: 308563878 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-510d5-predictor-8556d94969-wh62f |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-510d5-predictor-8556d94969-wh62f |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-510d5-predictor-8556d94969-wh62f |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.198s (2.198s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-510d5-predictor-5ccb9788d8-cqw5l |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-510d5-predictor-5ccb9788d8-cqw5l |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.449s (2.449s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-510d5-predictor-5ccb9788d8-cqw5l |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-ksclf |
Pulling |
Pulling image "kserve/xgbserver:latest" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5679f69dd7-ktwmr |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-869f8dd9fc-vpddx |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-869f8dd9fc-vpddx |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1968" in 15.709s (15.709s including waiting). Image size: 1583085459 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5679f69dd7-ktwmr |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1968" in 15.703s (15.703s including waiting). Image size: 1583085459 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5679f69dd7-ktwmr |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5679f69dd7-ktwmr |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-869f8dd9fc-vpddx |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-869f8dd9fc-vpddx |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5679f69dd7-ktwmr |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.441s (2.441s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5679f69dd7-ktwmr |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-869f8dd9fc-vpddx |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.447s (2.447s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-869f8dd9fc-vpddx |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5679f69dd7-ktwmr |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-869f8dd9fc-vpddx |
Created |
Created container: kube-rbac-proxy | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-ksclf |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-ksclf |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-ksclf |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-ksclf |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-ksclf |
Pulled |
Successfully pulled image "kserve/xgbserver:latest" in 18.009s (18.009s including waiting). Image size: 1332293374 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-ksclf |
Started |
Started container kserve-container | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-510d5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-510d5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-510d5-predictor-8556d94969-wh62f |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.26:8080: connect: connection refused |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-510d5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-510d5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-510d5 |
InferenceServiceReady |
InferenceService [error-404-isvc-510d5] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-510d5 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-510d5 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-510d5 |
InferenceServiceReady |
InferenceService [success-200-isvc-510d5] is Ready | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-510d5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-510d5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
multus |
switch-graph-510d5-85cb7dd998-8x2xs |
AddedInterface |
Add eth0 [10.133.0.28/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-510d5-85cb7dd998-8x2xs |
Pulling |
Pulling image "quay.io/opendatahub/kserve-router@sha256:e8d0e1738b2d4bc216fc02035501d68a4ec15b147a5f84c21018b5b4330d1491" | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-510d5-85cb7dd998 |
SuccessfulCreate |
Created pod: switch-graph-510d5-85cb7dd998-8x2xs | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-510d5 |
ScalingReplicaSet |
Scaled up replica set switch-graph-510d5-85cb7dd998 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-510d5 |
InferenceGraphReady |
InferenceGraph [switch-graph-510d5] is Ready | |
kserve-ci-e2e-test |
kubelet |
switch-graph-510d5-85cb7dd998-8x2xs |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-router@sha256:e8d0e1738b2d4bc216fc02035501d68a4ec15b147a5f84c21018b5b4330d1491" in 1.97s (1.97s including waiting). Image size: 215912525 bytes. | |
kserve-ci-e2e-test |
kubelet |
switch-graph-510d5-85cb7dd998-8x2xs |
Created |
Created container: switch-graph-510d5 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-510d5-85cb7dd998-8x2xs |
Started |
Started container switch-graph-510d5 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-510d5-predictor-8556d94969-wh62f |
Killing |
Stopping container kube-rbac-proxy | |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-510d5-predictor-5ccb9788d8-cqw5l |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-510d5-predictor-5ccb9788d8-cqw5l |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-2e273-predictor-85c84c9849 |
SuccessfulCreate |
Created pod: success-200-isvc-2e273-predictor-85c84c9849-pcfd9 | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-2e273-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-2e273-predictor-85c84c9849 from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-2e273-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-2e273-predictor-6db6bf4f6 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-2e273-predictor-6db6bf4f6 |
SuccessfulCreate |
Created pod: error-404-isvc-2e273-predictor-6db6bf4f6-6q6b2 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-510d5-predictor-8556d94969-wh62f |
Killing |
Stopping container kserve-container | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-510d5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-510d5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
switch-graph-510d5-85cb7dd998-8x2xs |
Killing |
Stopping container switch-graph-510d5 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e273-predictor-6db6bf4f6-6q6b2 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e273-predictor-85c84c9849-pcfd9 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e273-predictor-85c84c9849-pcfd9 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1968" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-2e273-predictor-6db6bf4f6-6q6b2 |
AddedInterface |
Add eth0 [10.132.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e273-predictor-6db6bf4f6-6q6b2 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1968" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e273-predictor-6db6bf4f6-6q6b2 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e273-predictor-6db6bf4f6-6q6b2 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e273-predictor-6db6bf4f6-6q6b2 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e273-predictor-85c84c9849-pcfd9 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-2e273-predictor-85c84c9849-pcfd9 |
AddedInterface |
Add eth0 [10.133.0.29/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e273-predictor-6db6bf4f6-6q6b2 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e273-predictor-85c84c9849-pcfd9 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e273-predictor-85c84c9849-pcfd9 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e273-predictor-85c84c9849-pcfd9 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-510d5-predictor-5ccb9788d8-cqw5l |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.31:8643/healthz": dial tcp 10.132.0.31:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-510d5-predictor-5ccb9788d8-cqw5l |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.31:8080: connect: connection refused |
| (x8) | kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-ksclf |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.27:8080: connect: connection refused |
| (x10) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5679f69dd7-ktwmr |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.18:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-1] is Ready | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
InferenceServiceReady |
InferenceService [isvc-xgboost-graph] is Ready | |
| (x10) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-2] is Ready | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-510d5-85cb7dd998-8x2xs |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e273-predictor-85c84c9849-pcfd9 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.29:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
model-chainer-6f496df4c6-8w6hl |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:e8d0e1738b2d4bc216fc02035501d68a4ec15b147a5f84c21018b5b4330d1491" already present on machine | |
kserve-ci-e2e-test |
kubelet |
model-chainer-6f496df4c6-8w6hl |
Created |
Created container: model-chainer | |
kserve-ci-e2e-test |
deployment-controller |
model-chainer |
ScalingReplicaSet |
Scaled up replica set model-chainer-6f496df4c6 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
model-chainer-6f496df4c6 |
SuccessfulCreate |
Created pod: model-chainer-6f496df4c6-8w6hl | |
kserve-ci-e2e-test |
InferenceGraphController |
model-chainer |
InferenceGraphReady |
InferenceGraph [model-chainer] is Ready | |
kserve-ci-e2e-test |
kubelet |
model-chainer-6f496df4c6-8w6hl |
Started |
Started container model-chainer | |
kserve-ci-e2e-test |
multus |
model-chainer-6f496df4c6-8w6hl |
AddedInterface |
Add eth0 [10.133.0.30/23] from ovn-kubernetes | |
| (x7) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x7) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2e273-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2e273-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2e273-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2e273-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-2e273 |
InferenceServiceReady |
InferenceService [success-200-isvc-2e273] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-2e273 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-2e273 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-2e273 |
InferenceServiceReady |
InferenceService [error-404-isvc-2e273] is Ready | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-869f8dd9fc-vpddx |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-869f8dd9fc-vpddx |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5679f69dd7-ktwmr |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5679f69dd7-ktwmr |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-53cb1-predictor-7578994fd6 |
SuccessfulCreate |
Created pod: error-404-isvc-53cb1-predictor-7578994fd6-m8xvq | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-53cb1-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-53cb1-predictor-7578994fd6 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
model-chainer-6f496df4c6-8w6hl |
Killing |
Stopping container model-chainer | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-ksclf |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-53cb1-predictor-77bd468459-7gnsg |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-53cb1-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-53cb1-predictor-77bd468459 |
SuccessfulCreate |
Created pod: success-200-isvc-53cb1-predictor-77bd468459-7gnsg | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-53cb1-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-53cb1-predictor-77bd468459 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-ksclf |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-53cb1-predictor-77bd468459-7gnsg |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-53cb1-predictor-77bd468459-7gnsg |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-53cb1-predictor-77bd468459-7gnsg |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-53cb1-predictor-77bd468459-7gnsg |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1968" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-53cb1-predictor-77bd468459-7gnsg |
AddedInterface |
Add eth0 [10.133.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5679f69dd7-ktwmr |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.18:8643/healthz": dial tcp 10.134.0.18:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-53cb1-predictor-7578994fd6-m8xvq |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-53cb1-predictor-7578994fd6-m8xvq |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-53cb1-predictor-7578994fd6-m8xvq |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-53cb1-predictor-7578994fd6-m8xvq |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-53cb1-predictor-77bd468459-7gnsg |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-53cb1-predictor-77bd468459-7gnsg |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-53cb1-predictor-7578994fd6-m8xvq |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-53cb1-predictor-7578994fd6-m8xvq |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1968" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-53cb1-predictor-7578994fd6-m8xvq |
AddedInterface |
Add eth0 [10.132.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-869f8dd9fc-vpddx |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.17:8643/healthz": dial tcp 10.134.0.17:8643: connect: connection refused | |
| (x11) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-869f8dd9fc-vpddx |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.17:8080: connect: connection refused |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-2e273-55d76c96b5 |
SuccessfulCreate |
Created pod: switch-graph-2e273-55d76c96b5-pbttp | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-2e273 |
ScalingReplicaSet |
Scaled up replica set switch-graph-2e273-55d76c96b5 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-2e273 |
InferenceGraphReady |
InferenceGraph [switch-graph-2e273] is Ready | |
kserve-ci-e2e-test |
kubelet |
switch-graph-2e273-55d76c96b5-pbttp |
Started |
Started container switch-graph-2e273 | |
kserve-ci-e2e-test |
multus |
switch-graph-2e273-55d76c96b5-pbttp |
AddedInterface |
Add eth0 [10.133.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-2e273-55d76c96b5-pbttp |
Created |
Created container: switch-graph-2e273 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-2e273-55d76c96b5-pbttp |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:e8d0e1738b2d4bc216fc02035501d68a4ec15b147a5f84c21018b5b4330d1491" already present on machine | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2e273-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-2e273 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-2e273 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2e273-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2e273-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2e273-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
model-chainer-6f496df4c6-8w6hl |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-53cb1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-53cb1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-53cb1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-53cb1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-53cb1-predictor-7578994fd6-m8xvq |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.33:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-53cb1-predictor-77bd468459-7gnsg |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.31:8080: connect: connection refused |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-2e273 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-2e273 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-53cb1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-53cb1 |
InferenceServiceReady |
InferenceService [error-404-isvc-53cb1] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-53cb1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-53cb1 |
InferenceServiceReady |
InferenceService [success-200-isvc-53cb1] is Ready | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-53cb1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-53cb1 |
InferenceGraphReady |
InferenceGraph [sequence-graph-53cb1] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-53cb1 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-53cb1-d889798ff from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-53cb1-d889798ff |
SuccessfulCreate |
Created pod: sequence-graph-53cb1-d889798ff-6bkzb | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-53cb1-d889798ff-6bkzb |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:e8d0e1738b2d4bc216fc02035501d68a4ec15b147a5f84c21018b5b4330d1491" already present on machine | |
kserve-ci-e2e-test |
multus |
sequence-graph-53cb1-d889798ff-6bkzb |
AddedInterface |
Add eth0 [10.133.0.33/23] from ovn-kubernetes | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-53cb1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-53cb1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-53cb1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
sequence-graph-53cb1-d889798ff-6bkzb |
Started |
Started container sequence-graph-53cb1 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-53cb1-d889798ff-6bkzb |
Created |
Created container: sequence-graph-53cb1 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-53cb1 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-53cb1 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e273-predictor-6db6bf4f6-6q6b2 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-b3d7b-predictor-544b6dfd74-mbzlt |
AddedInterface |
Add eth0 [10.132.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-b3d7b-predictor-544b6dfd74 |
SuccessfulCreate |
Created pod: error-404-isvc-b3d7b-predictor-544b6dfd74-mbzlt | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-b3d7b-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-b3d7b-predictor-544b6dfd74 from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-b3d7b |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-b3d7b": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-b3d7b": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-b3d7b |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-b3d7b": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-b3d7b |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-b3d7b": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-b3d7b": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-b3d7b |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-b3d7b": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e273-predictor-6db6bf4f6-6q6b2 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b3d7b-predictor-5b7c8bf675-xwlk6 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-b3d7b-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b3d7b-predictor-544b6dfd74-mbzlt |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1968" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e273-predictor-85c84c9849-pcfd9 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-b3d7b-predictor-5b7c8bf675 |
SuccessfulCreate |
Created pod: success-200-isvc-b3d7b-predictor-5b7c8bf675-xwlk6 | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-b3d7b-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-b3d7b-predictor-5b7c8bf675 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-2e273-55d76c96b5-pbttp |
Killing |
Stopping container switch-graph-2e273 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e273-predictor-85c84c9849-pcfd9 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b3d7b-predictor-5b7c8bf675-xwlk6 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b3d7b-predictor-5b7c8bf675-xwlk6 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b3d7b-predictor-544b6dfd74-mbzlt |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-b3d7b-predictor-5b7c8bf675-xwlk6 |
AddedInterface |
Add eth0 [10.133.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b3d7b-predictor-5b7c8bf675-xwlk6 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1968" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b3d7b-predictor-544b6dfd74-mbzlt |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b3d7b-predictor-5b7c8bf675-xwlk6 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b3d7b-predictor-5b7c8bf675-xwlk6 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b3d7b-predictor-544b6dfd74-mbzlt |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b3d7b-predictor-544b6dfd74-mbzlt |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b3d7b-predictor-5b7c8bf675-xwlk6 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b3d7b-predictor-544b6dfd74-mbzlt |
Started |
Started container kserve-container | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e273-predictor-6db6bf4f6-6q6b2 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.32:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e273-predictor-6db6bf4f6-6q6b2 |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.32:8643/healthz": dial tcp 10.132.0.32:8643: connect: connection refused | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-b3d7b-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-b3d7b-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-2e273-55d76c96b5-pbttp |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-b3d7b-predictor-5b7c8bf675-xwlk6 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.34:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-b3d7b-predictor-544b6dfd74-mbzlt |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.34:8080: connect: connection refused |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-b3d7b-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-b3d7b-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-b3d7b |
InferenceServiceReady |
InferenceService [success-200-isvc-b3d7b] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-b3d7b |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-b3d7b |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-b3d7b |
InferenceServiceReady |
InferenceService [error-404-isvc-b3d7b] is Ready | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-53cb1-predictor-7578994fd6-m8xvq |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-90077-predictor-58445bbccc-n7mjk |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-90077-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-90077-predictor-7767f7d476 |
SuccessfulCreate |
Created pod: error-404-isvc-90077-predictor-7767f7d476-wsrwq | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-90077-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-90077-predictor-7767f7d476 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-53cb1-d889798ff-6bkzb |
Killing |
Stopping container sequence-graph-53cb1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-90077-predictor-58445bbccc |
SuccessfulCreate |
Created pod: success-200-isvc-90077-predictor-58445bbccc-n7mjk | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-53cb1-predictor-77bd468459-7gnsg |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-53cb1-predictor-77bd468459-7gnsg |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-90077-predictor-7767f7d476-wsrwq |
AddedInterface |
Add eth0 [10.132.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-53cb1-predictor-7578994fd6-m8xvq |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-90077-predictor-7767f7d476-wsrwq |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1968" already present on machine | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-90077-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-90077-predictor-58445bbccc from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-90077-predictor-7767f7d476-wsrwq |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-90077-predictor-58445bbccc-n7mjk |
AddedInterface |
Add eth0 [10.133.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-90077-predictor-58445bbccc-n7mjk |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-90077-predictor-58445bbccc-n7mjk |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-90077-predictor-7767f7d476-wsrwq |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-90077-predictor-58445bbccc-n7mjk |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-90077-predictor-58445bbccc-n7mjk |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-90077-predictor-58445bbccc-n7mjk |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-90077-predictor-7767f7d476-wsrwq |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-90077-predictor-7767f7d476-wsrwq |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-90077-predictor-7767f7d476-wsrwq |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-90077-predictor-58445bbccc-n7mjk |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1968" already present on machine | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-b3d7b-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-b3d7b |
InferenceGraphReady |
InferenceGraph [ensemble-graph-b3d7b] is Ready | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-b3d7b-6c989bffcf-kgvbk |
Started |
Started container ensemble-graph-b3d7b | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-b3d7b-6c989bffcf-kgvbk |
Created |
Created container: ensemble-graph-b3d7b | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-b3d7b-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-b3d7b-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-b3d7b-6c989bffcf-kgvbk |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:e8d0e1738b2d4bc216fc02035501d68a4ec15b147a5f84c21018b5b4330d1491" already present on machine | |
kserve-ci-e2e-test |
multus |
ensemble-graph-b3d7b-6c989bffcf-kgvbk |
AddedInterface |
Add eth0 [10.133.0.36/23] from ovn-kubernetes | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-b3d7b-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-b3d7b-6c989bffcf |
SuccessfulCreate |
Created pod: ensemble-graph-b3d7b-6c989bffcf-kgvbk | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-b3d7b |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-b3d7b-6c989bffcf from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b3d7b-predictor-5b7c8bf675-xwlk6 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-81279-predictor-bb8f78f5c-qlsqq |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-81279-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-81279-predictor-bb8f78f5c |
SuccessfulCreate |
Created pod: success-200-isvc-81279-predictor-bb8f78f5c-qlsqq | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-81279-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-81279-predictor-bb8f78f5c from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-81279-predictor-84575cb8cb-pllk2 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-81279-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b3d7b-predictor-544b6dfd74-mbzlt |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b3d7b-predictor-544b6dfd74-mbzlt |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b3d7b-predictor-5b7c8bf675-xwlk6 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-b3d7b-6c989bffcf-kgvbk |
Killing |
Stopping container ensemble-graph-b3d7b | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-81279-predictor-84575cb8cb |
SuccessfulCreate |
Created pod: error-404-isvc-81279-predictor-84575cb8cb-pllk2 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-81279-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-81279-predictor-84575cb8cb from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-81279-predictor-bb8f78f5c-qlsqq |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-81279-predictor-bb8f78f5c-qlsqq |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-81279-predictor-84575cb8cb-pllk2 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-81279-predictor-84575cb8cb-pllk2 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
success-200-isvc-81279-predictor-bb8f78f5c-qlsqq |
AddedInterface |
Add eth0 [10.133.0.37/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-81279-predictor-bb8f78f5c-qlsqq |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-81279-predictor-84575cb8cb-pllk2 |
AddedInterface |
Add eth0 [10.132.0.36/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-81279-predictor-bb8f78f5c-qlsqq |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-81279-predictor-84575cb8cb-pllk2 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-81279-predictor-84575cb8cb-pllk2 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-81279-predictor-84575cb8cb-pllk2 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1968" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-81279-predictor-bb8f78f5c-qlsqq |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-81279-predictor-84575cb8cb-pllk2 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-81279-predictor-bb8f78f5c-qlsqq |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1968" already present on machine | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-53cb1-d889798ff-6bkzb |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-90077-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-90077-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-90077-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-90077-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-90077-predictor-7767f7d476-wsrwq |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.35:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-90077-predictor-58445bbccc-n7mjk |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.35:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-b3d7b-6c989bffcf-kgvbk |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-90077 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-90077 |
InferenceServiceReady |
InferenceService [error-404-isvc-90077] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-90077 |
InferenceServiceReady |
InferenceService [success-200-isvc-90077] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-90077 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-81279-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-81279-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-81279-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-81279-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-81279-predictor-84575cb8cb-pllk2 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.36:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-81279-predictor-bb8f78f5c-qlsqq |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.37:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
sequence-graph-90077-7b4844c498-zld4d |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "sequence-graph-90077-serving-cert" not found | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-90077-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-90077 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-90077-7b4844c498 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-90077-7b4844c498 |
SuccessfulCreate |
Created pod: sequence-graph-90077-7b4844c498-zld4d | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-90077 |
InferenceGraphReady |
InferenceGraph [sequence-graph-90077] is Ready | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-90077-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-90077-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-90077-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
sequence-graph-90077-7b4844c498-zld4d |
Started |
Started container sequence-graph-90077 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-90077-7b4844c498-zld4d |
Created |
Created container: sequence-graph-90077 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-90077-7b4844c498-zld4d |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:e8d0e1738b2d4bc216fc02035501d68a4ec15b147a5f84c21018b5b4330d1491" already present on machine | |
kserve-ci-e2e-test |
multus |
sequence-graph-90077-7b4844c498-zld4d |
AddedInterface |
Add eth0 [10.133.0.38/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-81279 |
InferenceServiceReady |
InferenceService [error-404-isvc-81279] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-81279 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-81279 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-81279 |
InferenceServiceReady |
InferenceService [success-200-isvc-81279] is Ready | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-28e88-predictor-6bb6f499b4-jg8ns |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-28e88-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-90077-predictor-7767f7d476-wsrwq |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-90077-7b4844c498-zld4d |
Killing |
Stopping container sequence-graph-90077 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-90077-predictor-58445bbccc-n7mjk |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-90077-predictor-58445bbccc-n7mjk |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-28e88 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-28e88": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-28e88": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-28e88 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-28e88": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-28e88-predictor-6bb6f499b4 |
SuccessfulCreate |
Created pod: error-404-isvc-28e88-predictor-6bb6f499b4-jg8ns | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-28e88-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-28e88-predictor-779f848669 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-28e88-predictor-779f848669 |
SuccessfulCreate |
Created pod: success-200-isvc-28e88-predictor-779f848669-jfwgg | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-90077-predictor-7767f7d476-wsrwq |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-28e88-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-28e88-predictor-6bb6f499b4 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-28e88-predictor-6bb6f499b4-jg8ns |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1968" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-28e88-predictor-6bb6f499b4-jg8ns |
AddedInterface |
Add eth0 [10.132.0.37/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-28e88-predictor-779f848669-jfwgg |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-28e88-predictor-779f848669-jfwgg |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-28e88-predictor-779f848669-jfwgg |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-28e88-predictor-779f848669-jfwgg |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-28e88-predictor-779f848669-jfwgg |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1968" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-28e88-predictor-6bb6f499b4-jg8ns |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-28e88-predictor-6bb6f499b4-jg8ns |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-28e88-predictor-779f848669-jfwgg |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
success-200-isvc-28e88-predictor-779f848669-jfwgg |
AddedInterface |
Add eth0 [10.133.0.39/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-28e88-predictor-6bb6f499b4-jg8ns |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-28e88-predictor-6bb6f499b4-jg8ns |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-28e88-predictor-6bb6f499b4-jg8ns |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-90077-predictor-7767f7d476-wsrwq |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.35:8643/healthz": dial tcp 10.132.0.35:8643: connect: connection refused | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-81279 |
InferenceGraphReady |
InferenceGraph [ensemble-graph-81279] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-81279 |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-81279-7ff94b5fb from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-81279-7ff94b5fb |
SuccessfulCreate |
Created pod: ensemble-graph-81279-7ff94b5fb-vhq95 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-81279-7ff94b5fb-vhq95 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "ensemble-graph-81279-serving-cert" not found | |
kserve-ci-e2e-test |
multus |
ensemble-graph-81279-7ff94b5fb-vhq95 |
AddedInterface |
Add eth0 [10.133.0.40/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-81279-7ff94b5fb-vhq95 |
Started |
Started container ensemble-graph-81279 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-81279-7ff94b5fb-vhq95 |
Created |
Created container: ensemble-graph-81279 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-81279-7ff94b5fb-vhq95 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:e8d0e1738b2d4bc216fc02035501d68a4ec15b147a5f84c21018b5b4330d1491" already present on machine | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-81279-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-81279 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-81279-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-81279-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-81279 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-81279-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-90077-7b4844c498-zld4d |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-28e88-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-28e88-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-28e88-predictor-6bb6f499b4-jg8ns |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.37:8080: connect: connection refused |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-81279 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-81279 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-28e88-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-28e88-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-28e88 |
InferenceServiceReady |
InferenceService [success-200-isvc-28e88] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-28e88 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-28e88 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-28e88 |
InferenceServiceReady |
InferenceService [error-404-isvc-28e88] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-28e88-f549854fc |
SuccessfulCreate |
Created pod: sequence-graph-28e88-f549854fc-7qtn8 | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-28e88 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-28e88-f549854fc from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-28e88 |
InferenceGraphReady |
InferenceGraph [sequence-graph-28e88] is Ready | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-28e88-f549854fc-7qtn8 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:e8d0e1738b2d4bc216fc02035501d68a4ec15b147a5f84c21018b5b4330d1491" already present on machine | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-28e88-f549854fc-7qtn8 |
Created |
Created container: sequence-graph-28e88 | |
kserve-ci-e2e-test |
multus |
sequence-graph-28e88-f549854fc-7qtn8 |
AddedInterface |
Add eth0 [10.133.0.41/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-28e88-f549854fc-7qtn8 |
Started |
Started container sequence-graph-28e88 | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-28e88 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-28e88 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-28e88-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-28e88-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-28e88-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-28e88-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-28e88 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-28e88 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-326f5-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-326f5-predictor-5f77687698 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-81279-predictor-84575cb8cb-pllk2 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-81279-predictor-bb8f78f5c-qlsqq |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-326f5-predictor-5f77687698-df8ms |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-326f5-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-81279-predictor-84575cb8cb-pllk2 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-81279-predictor-bb8f78f5c-qlsqq |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-326f5-predictor-5f77687698 |
SuccessfulCreate |
Created pod: error-404-isvc-326f5-predictor-5f77687698-df8ms | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-326f5-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-326f5-predictor-669c54dd8c from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-326f5-predictor-669c54dd8c |
SuccessfulCreate |
Created pod: success-200-isvc-326f5-predictor-669c54dd8c-85sxw | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-81279-7ff94b5fb-vhq95 |
Killing |
Stopping container ensemble-graph-81279 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-326f5-predictor-669c54dd8c-85sxw |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-326f5-predictor-669c54dd8c-85sxw |
AddedInterface |
Add eth0 [10.133.0.42/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-326f5-predictor-5f77687698-df8ms |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-326f5-predictor-5f77687698-df8ms |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-326f5-predictor-5f77687698-df8ms |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-326f5-predictor-669c54dd8c-85sxw |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-326f5-predictor-5f77687698-df8ms |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-326f5-predictor-5f77687698-df8ms |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1968" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-326f5-predictor-5f77687698-df8ms |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-326f5-predictor-5f77687698-df8ms |
AddedInterface |
Add eth0 [10.132.0.38/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-326f5-predictor-669c54dd8c-85sxw |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-326f5-predictor-669c54dd8c-85sxw |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1968" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-326f5-predictor-669c54dd8c-85sxw |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-326f5-predictor-669c54dd8c-85sxw |
Created |
Created container: kserve-container | |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-81279-7ff94b5fb-vhq95 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-326f5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-326f5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-326f5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-326f5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-326f5-predictor-5f77687698-df8ms |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.38:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-326f5-predictor-669c54dd8c-85sxw |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.42:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-326f5 |
InferenceServiceReady |
InferenceService [error-404-isvc-326f5] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-326f5 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-326f5 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-326f5 |
InferenceServiceReady |
InferenceService [success-200-isvc-326f5] is Ready | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-28e88-predictor-6bb6f499b4-jg8ns |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6a9b1-predictor-8496c76f6d-6c6kk |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-6a9b1-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-6a9b1-predictor-7bc68bf869 |
SuccessfulCreate |
Created pod: success-200-isvc-6a9b1-predictor-7bc68bf869-7z2fw | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6a9b1-predictor-7bc68bf869-7z2fw |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1968" already present on machine | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-6a9b1 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-6a9b1": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-6a9b1": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-6a9b1 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-6a9b1": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
multus |
success-200-isvc-6a9b1-predictor-7bc68bf869-7z2fw |
AddedInterface |
Add eth0 [10.133.0.43/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-6a9b1-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-6a9b1-predictor-7bc68bf869 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-28e88-predictor-779f848669-jfwgg |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-28e88-predictor-779f848669-jfwgg |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-28e88-f549854fc-7qtn8 |
Killing |
Stopping container sequence-graph-28e88 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-6a9b1-predictor-8496c76f6d |
SuccessfulCreate |
Created pod: error-404-isvc-6a9b1-predictor-8496c76f6d-6c6kk | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-6a9b1-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-6a9b1-predictor-8496c76f6d from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-28e88-predictor-6bb6f499b4-jg8ns |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6a9b1-predictor-8496c76f6d-6c6kk |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6a9b1-predictor-8496c76f6d-6c6kk |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1968" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-6a9b1-predictor-8496c76f6d-6c6kk |
AddedInterface |
Add eth0 [10.132.0.39/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6a9b1-predictor-8496c76f6d-6c6kk |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6a9b1-predictor-8496c76f6d-6c6kk |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6a9b1-predictor-7bc68bf869-7z2fw |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6a9b1-predictor-8496c76f6d-6c6kk |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6a9b1-predictor-8496c76f6d-6c6kk |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6a9b1-predictor-7bc68bf869-7z2fw |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6a9b1-predictor-7bc68bf869-7z2fw |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6a9b1-predictor-7bc68bf869-7z2fw |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6a9b1-predictor-7bc68bf869-7z2fw |
Created |
Created container: kube-rbac-proxy | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-28e88-predictor-779f848669-jfwgg |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.39:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-28e88-predictor-779f848669-jfwgg |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.39:8643/healthz": dial tcp 10.133.0.39:8643: connect: connection refused | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-326f5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-326f5-768586467d |
SuccessfulCreate |
Created pod: splitter-graph-326f5-768586467d-gw8kd | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-326f5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-326f5 |
InferenceGraphReady |
InferenceGraph [splitter-graph-326f5] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-326f5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-326f5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-326f5 |
ScalingReplicaSet |
Scaled up replica set splitter-graph-326f5-768586467d from 0 to 1 | |
kserve-ci-e2e-test |
multus |
splitter-graph-326f5-768586467d-gw8kd |
AddedInterface |
Add eth0 [10.133.0.44/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-326f5-768586467d-gw8kd |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:e8d0e1738b2d4bc216fc02035501d68a4ec15b147a5f84c21018b5b4330d1491" already present on machine | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-326f5-768586467d-gw8kd |
Created |
Created container: splitter-graph-326f5 | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-326f5-768586467d-gw8kd |
Started |
Started container splitter-graph-326f5 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-7dad6-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-7dad6-predictor-6c8d884d74 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-326f5-predictor-669c54dd8c-85sxw |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-326f5-768586467d-gw8kd |
Killing |
Stopping container splitter-graph-326f5 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-326f5-predictor-669c54dd8c-85sxw |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-7dad6-predictor-7c8fc894cf |
SuccessfulCreate |
Created pod: success-200-isvc-7dad6-predictor-7c8fc894cf-46kb9 | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-7dad6-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-7dad6-predictor-7c8fc894cf from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-7dad6-predictor-6c8d884d74 |
SuccessfulCreate |
Created pod: error-404-isvc-7dad6-predictor-6c8d884d74-7hlfm | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-326f5-predictor-5f77687698-df8ms |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-326f5-predictor-5f77687698-df8ms |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-7dad6-predictor-6c8d884d74-7hlfm |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-7dad6-predictor-6c8d884d74-7hlfm |
AddedInterface |
Add eth0 [10.132.0.40/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-7dad6-predictor-6c8d884d74-7hlfm |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-7dad6-predictor-7c8fc894cf-46kb9 |
AddedInterface |
Add eth0 [10.133.0.45/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-7dad6-predictor-6c8d884d74-7hlfm |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-7dad6-predictor-7c8fc894cf-46kb9 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1968" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-7dad6-predictor-6c8d884d74-7hlfm |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-7dad6-predictor-6c8d884d74-7hlfm |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-7dad6-predictor-7c8fc894cf-46kb9 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-7dad6-predictor-7c8fc894cf-46kb9 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-7dad6-predictor-6c8d884d74-7hlfm |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1968" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-7dad6-predictor-7c8fc894cf-46kb9 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-7dad6-predictor-7c8fc894cf-46kb9 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-7dad6-predictor-7c8fc894cf-46kb9 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-326f5-predictor-5f77687698-df8ms |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.38:8643/healthz": dial tcp 10.132.0.38:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-326f5-predictor-669c54dd8c-85sxw |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.42:8643/healthz": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-28e88-f549854fc-7qtn8 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-6a9b1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-6a9b1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-6a9b1-predictor-7bc68bf869-7z2fw |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.43:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-6a9b1-predictor-8496c76f6d-6c6kk |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.39:8080: connect: connection refused |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-6a9b1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-6a9b1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-6a9b1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-6a9b1 |
InferenceServiceReady |
InferenceService [success-200-isvc-6a9b1] is Ready | |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-326f5-768586467d-gw8kd |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-6a9b1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-6a9b1 |
InferenceServiceReady |
InferenceService [error-404-isvc-6a9b1] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-7dad6-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-7dad6-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-7dad6-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-7dad6-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-7dad6-predictor-6c8d884d74-7hlfm |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.40:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-7dad6-predictor-7c8fc894cf-46kb9 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.45:8080: connect: connection refused |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-6a9b1 |
ScalingReplicaSet |
Scaled up replica set switch-graph-6a9b1-6d87596d7f from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-6a9b1 |
InferenceGraphReady |
InferenceGraph [switch-graph-6a9b1] is Ready | |
kserve-ci-e2e-test |
kubelet |
switch-graph-6a9b1-6d87596d7f-pdg8z |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-6a9b1-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-6a9b1-6d87596d7f |
SuccessfulCreate |
Created pod: switch-graph-6a9b1-6d87596d7f-pdg8z | |
kserve-ci-e2e-test |
multus |
switch-graph-6a9b1-6d87596d7f-pdg8z |
AddedInterface |
Add eth0 [10.133.0.46/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-6a9b1-6d87596d7f-pdg8z |
Created |
Created container: switch-graph-6a9b1 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-6a9b1-6d87596d7f-pdg8z |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:e8d0e1738b2d4bc216fc02035501d68a4ec15b147a5f84c21018b5b4330d1491" already present on machine | |
kserve-ci-e2e-test |
kubelet |
switch-graph-6a9b1-6d87596d7f-pdg8z |
Started |
Started container switch-graph-6a9b1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-7dad6 |
InferenceServiceReady |
InferenceService [error-404-isvc-7dad6] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-7dad6 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-7dad6 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-7dad6 |
InferenceServiceReady |
InferenceService [success-200-isvc-7dad6] is Ready | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-6a9b1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-6a9b1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-7dad6-75dfbcc7d7 |
SuccessfulCreate |
Created pod: splitter-graph-7dad6-75dfbcc7d7-nd9vt | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-7dad6 |
ScalingReplicaSet |
Scaled up replica set splitter-graph-7dad6-75dfbcc7d7 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-7dad6 |
InferenceGraphReady |
InferenceGraph [splitter-graph-7dad6] is Ready | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-7dad6-75dfbcc7d7-nd9vt |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:e8d0e1738b2d4bc216fc02035501d68a4ec15b147a5f84c21018b5b4330d1491" already present on machine | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-7dad6-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
splitter-graph-7dad6-75dfbcc7d7-nd9vt |
Started |
Started container splitter-graph-7dad6 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-7dad6-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
multus |
splitter-graph-7dad6-75dfbcc7d7-nd9vt |
AddedInterface |
Add eth0 [10.133.0.47/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-7dad6-75dfbcc7d7-nd9vt |
Created |
Created container: splitter-graph-7dad6 | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-6a9b1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-6a9b1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-7dad6-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-7dad6-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-7dad6 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-7dad6 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-6a9b1 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-6a9b1 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-7dad6-predictor-6c8d884d74-7hlfm |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-7dad6-predictor-7c8fc894cf-46kb9 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-7dad6-predictor-6c8d884d74-7hlfm |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-7dad6-predictor-7c8fc894cf-46kb9 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-7dad6-75dfbcc7d7-nd9vt |
Killing |
Stopping container splitter-graph-7dad6 | |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-7dad6-75dfbcc7d7-nd9vt |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
switch-graph-6a9b1-6d87596d7f-pdg8z |
Killing |
Stopping container switch-graph-6a9b1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6a9b1-predictor-8496c76f6d-6c6kk |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-6a9b1-predictor-8496c76f6d-6c6kk |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6a9b1-predictor-7bc68bf869-7z2fw |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6a9b1-predictor-7bc68bf869-7z2fw |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-6a9b1-predictor-7bc68bf869-7z2fw |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.43:8643/healthz": dial tcp 10.133.0.43:8643: connect: connection refused | |
| (x5) | kserve-ci-e2e-test |
kubelet |
switch-graph-6a9b1-6d87596d7f-pdg8z |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |