| Time | Namespace | Component | RelatedObject | Reason | Message |
|---|---|---|---|---|---|
kserve-ci-e2e-test |
switch-graph-87be3-bdb646768-z4lgk |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-87be3-bdb646768-z4lgk to ip-10-0-133-71.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-13b40-predictor-8bb687fb7-g48nb |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-13b40-predictor-8bb687fb7-g48nb to ip-10-0-140-239.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-7d014-7f8d646646-jnp9w |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-7d014-7f8d646646-jnp9w to ip-10-0-133-71.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-601fd-predictor-646cfc64df-ckjl2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-601fd-predictor-646cfc64df-ckjl2 to ip-10-0-140-239.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-13b40-64fb84556f-s67d4 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-13b40-64fb84556f-s67d4 to ip-10-0-133-71.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-e90df-764845479d-h54j2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-e90df-764845479d-h54j2 to ip-10-0-133-71.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-64898-predictor-57884f876f-8wq7r |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-64898-predictor-57884f876f-8wq7r to ip-10-0-140-239.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-c7da0-7b7778f46-tqcxt |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-c7da0-7b7778f46-tqcxt to ip-10-0-133-71.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-30f77-859fc5c779-w9th4 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-30f77-859fc5c779-w9th4 to ip-10-0-133-71.ec2.internal | ||
kserve-ci-e2e-test |
model-chainer-546cb8556c-kwp6s |
Scheduled |
Successfully assigned kserve-ci-e2e-test/model-chainer-546cb8556c-kwp6s to ip-10-0-133-71.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-e90df-predictor-5bbcb468bc-wkj7z |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-e90df-predictor-5bbcb468bc-wkj7z to ip-10-0-140-239.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-64898-76dcdb6787-r4g2b |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-64898-76dcdb6787-r4g2b to ip-10-0-133-71.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-ce1f6-predictor-76644689b9-smjt7 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-ce1f6-predictor-76644689b9-smjt7 to ip-10-0-140-239.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-c7da0-predictor-945f88485-8srks |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-c7da0-predictor-945f88485-8srks to ip-10-0-140-239.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-601fd-8f5c764c5-5vlqq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-601fd-8f5c764c5-5vlqq to ip-10-0-133-71.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-ce1f6-predictor-b77d84c64-v2lwk |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-ce1f6-predictor-b77d84c64-v2lwk to ip-10-0-141-177.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-87be3-predictor-845b844549-cpcg2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-87be3-predictor-845b844549-cpcg2 to ip-10-0-140-239.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-7d014-predictor-5688f58dbc-7pbff |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-7d014-predictor-5688f58dbc-7pbff to ip-10-0-140-239.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-e90df-predictor-5b7bc4c674-96vrn |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-e90df-predictor-5b7bc4c674-96vrn to ip-10-0-141-177.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-f794f-predictor-f4899d6f4-94s6d |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-f794f-predictor-f4899d6f4-94s6d to ip-10-0-141-177.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-87be3-predictor-5656fc75d4-x59s5 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-87be3-predictor-5656fc75d4-x59s5 to ip-10-0-141-177.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-1-predictor-7f476ffc7-bw8k9 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7f476ffc7-bw8k9 to ip-10-0-140-239.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-64898-predictor-c55974db4-6czmd |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-64898-predictor-c55974db4-6czmd to ip-10-0-141-177.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-7d014-predictor-57c444947d-llcg9 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-7d014-predictor-57c444947d-llcg9 to ip-10-0-141-177.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-13b40-predictor-5cf559cf78-8pgvv |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-13b40-predictor-5cf559cf78-8pgvv to ip-10-0-141-177.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-f794f-6766495c86-m57t4 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-f794f-6766495c86-m57t4 to ip-10-0-133-71.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-ce1f6-75dbfb64bd-q4pxj |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-ce1f6-75dbfb64bd-q4pxj to ip-10-0-133-71.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-30f77-predictor-6b79cc6889-k52x5 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-30f77-predictor-6b79cc6889-k52x5 to ip-10-0-140-239.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-2-predictor-766ff5986c-vkntk |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-766ff5986c-vkntk to ip-10-0-133-71.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-601fd-predictor-5978f84496-s6xpw |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-601fd-predictor-5978f84496-s6xpw to ip-10-0-141-177.ec2.internal | ||
kserve-ci-e2e-test |
isvc-xgboost-graph-predictor-555fc69d56-d85h2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-d85h2 to ip-10-0-140-239.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-30f77-predictor-6c9bbd784d-pwj9w |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-30f77-predictor-6c9bbd784d-pwj9w to ip-10-0-141-177.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-f794f-predictor-75b57f85bc-4s5nf |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-f794f-predictor-75b57f85bc-4s5nf to ip-10-0-140-239.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-c7da0-predictor-5488f9f968-qtw88 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-c7da0-predictor-5488f9f968-qtw88 to ip-10-0-141-177.ec2.internal | ||
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-1-predictor-7f476ffc7 |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-1-predictor-7f476ffc7-bw8k9 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-f794f-predictor-75b57f85bc |
SuccessfulCreate |
Created pod: success-200-isvc-f794f-predictor-75b57f85bc-4s5nf | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f794f-predictor-75b57f85bc-4s5nf |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-f794f-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-1-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-1-predictor-7f476ffc7 from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-f794f-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-f794f-predictor-75b57f85bc from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-f794f-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-f794f-predictor-f4899d6f4 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-2-predictor-766ff5986c |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-2-predictor-766ff5986c-vkntk | |
kserve-ci-e2e-test |
deployment-controller |
isvc-xgboost-graph-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-xgboost-graph-predictor-555fc69d56 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-f794f-predictor-f4899d6f4 |
SuccessfulCreate |
Created pod: error-404-isvc-f794f-predictor-f4899d6f4-94s6d | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-xgboost-graph-predictor-555fc69d56 |
SuccessfulCreate |
Created pod: isvc-xgboost-graph-predictor-555fc69d56-d85h2 | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-766ff5986c-vkntk |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "isvc-sklearn-graph-2-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
multus |
success-200-isvc-f794f-predictor-75b57f85bc-4s5nf |
AddedInterface |
Add eth0 [10.133.0.24/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f794f-predictor-f4899d6f4-94s6d |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-f794f-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-2-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-2-predictor-766ff5986c from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-d85h2 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "isvc-xgboost-graph-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7f476ffc7-bw8k9 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "isvc-sklearn-graph-1-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f794f-predictor-75b57f85bc-4s5nf |
Pulling |
Pulling image "quay.io/opendatahub/success-200-isvc:odh-pr-1946" | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-1-predictor-7f476ffc7-bw8k9 |
AddedInterface |
Add eth0 [10.133.0.25/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-d85h2 |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-d85h2 |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:5ea09478793377febcb6cdc9ccdf0f01773e1155c9dff7d28ca581c50565149f" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-f794f-predictor-f4899d6f4-94s6d |
AddedInterface |
Add eth0 [10.134.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7f476ffc7-bw8k9 |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:5ea09478793377febcb6cdc9ccdf0f01773e1155c9dff7d28ca581c50565149f" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7f476ffc7-bw8k9 |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7f476ffc7-bw8k9 |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-2-predictor-766ff5986c-vkntk |
AddedInterface |
Add eth0 [10.132.0.16/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f794f-predictor-f4899d6f4-94s6d |
Pulling |
Pulling image "quay.io/opendatahub/error-404-isvc:odh-pr-1946" | |
kserve-ci-e2e-test |
multus |
isvc-xgboost-graph-predictor-555fc69d56-d85h2 |
AddedInterface |
Add eth0 [10.133.0.26/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-d85h2 |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-766ff5986c-vkntk |
Pulling |
Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:5ea09478793377febcb6cdc9ccdf0f01773e1155c9dff7d28ca581c50565149f" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-766ff5986c-vkntk |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-766ff5986c-vkntk |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:5ea09478793377febcb6cdc9ccdf0f01773e1155c9dff7d28ca581c50565149f" in 3.033s (3.034s including waiting). Image size: 308572071 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-766ff5986c-vkntk |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7f476ffc7-bw8k9 |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1946" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-d85h2 |
Pulling |
Pulling image "kserve/xgbserver:latest" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-766ff5986c-vkntk |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1946" | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f794f-predictor-75b57f85bc-4s5nf |
Pulled |
Successfully pulled image "quay.io/opendatahub/success-200-isvc:odh-pr-1946" in 11.693s (11.693s including waiting). Image size: 1356695628 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f794f-predictor-75b57f85bc-4s5nf |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f794f-predictor-75b57f85bc-4s5nf |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f794f-predictor-75b57f85bc-4s5nf |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f794f-predictor-f4899d6f4-94s6d |
Pulled |
Successfully pulled image "quay.io/opendatahub/error-404-isvc:odh-pr-1946" in 13.681s (13.681s including waiting). Image size: 1355705921 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f794f-predictor-f4899d6f4-94s6d |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f794f-predictor-f4899d6f4-94s6d |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f794f-predictor-f4899d6f4-94s6d |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f794f-predictor-f4899d6f4-94s6d |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f794f-predictor-f4899d6f4-94s6d |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.2s (2.2s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f794f-predictor-f4899d6f4-94s6d |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7f476ffc7-bw8k9 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f794f-predictor-75b57f85bc-4s5nf |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 8.957s (8.957s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-766ff5986c-vkntk |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-766ff5986c-vkntk |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1946" in 12.87s (12.87s including waiting). Image size: 1583067020 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-766ff5986c-vkntk |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f794f-predictor-75b57f85bc-4s5nf |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7f476ffc7-bw8k9 |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1946" in 15.105s (15.105s including waiting). Image size: 1583067020 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7f476ffc7-bw8k9 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7f476ffc7-bw8k9 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7f476ffc7-bw8k9 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-766ff5986c-vkntk |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f794f-predictor-75b57f85bc-4s5nf |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7f476ffc7-bw8k9 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-766ff5986c-vkntk |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-766ff5986c-vkntk |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-766ff5986c-vkntk |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.533s (2.533s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-d85h2 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-d85h2 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-d85h2 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-d85h2 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-d85h2 |
Pulled |
Successfully pulled image "kserve/xgbserver:latest" in 23.727s (23.727s including waiting). Image size: 1332293375 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-d85h2 |
Started |
Started container kube-rbac-proxy | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f794f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f794f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f794f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f794f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-f794f-predictor-f4899d6f4-94s6d |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.35:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-f794f-predictor-75b57f85bc-4s5nf |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.24:8080: connect: connection refused |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f794f |
InferenceServiceReady |
InferenceService [error-404-isvc-f794f] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f794f |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-f794f |
InferenceServiceReady |
InferenceService [success-200-isvc-f794f] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-f794f |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f794f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f794f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-f794f |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "switch-graph-f794f": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-f794f |
UpdateFailed |
Failed to update status for InferenceGraph "switch-graph-f794f": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "switch-graph-f794f": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-f794f |
InferenceGraphReady |
InferenceGraph [switch-graph-f794f] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-f794f |
ScalingReplicaSet |
Scaled up replica set switch-graph-f794f-6766495c86 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-f794f-6766495c86 |
SuccessfulCreate |
Created pod: switch-graph-f794f-6766495c86-m57t4 | |
kserve-ci-e2e-test |
multus |
switch-graph-f794f-6766495c86-m57t4 |
AddedInterface |
Add eth0 [10.132.0.17/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-f794f-6766495c86-m57t4 |
Pulling |
Pulling image "quay.io/opendatahub/kserve-router@sha256:2ae619f4eea9fa406199e7b8e562bc31c295665fd082c5f377ac0602bcc5b2e9" | |
kserve-ci-e2e-test |
kubelet |
switch-graph-f794f-6766495c86-m57t4 |
Created |
Created container: switch-graph-f794f | |
kserve-ci-e2e-test |
kubelet |
switch-graph-f794f-6766495c86-m57t4 |
Started |
Started container switch-graph-f794f | |
kserve-ci-e2e-test |
kubelet |
switch-graph-f794f-6766495c86-m57t4 |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-router@sha256:2ae619f4eea9fa406199e7b8e562bc31c295665fd082c5f377ac0602bcc5b2e9" in 5.396s (5.396s including waiting). Image size: 214921806 bytes. | |
| (x8) | kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-d85h2 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.26:8080: connect: connection refused |
| (x9) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7f476ffc7-bw8k9 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.25:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f794f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f794f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
switch-graph-f794f-6766495c86-m57t4 |
Killing |
Stopping container switch-graph-f794f | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f794f-predictor-75b57f85bc-4s5nf |
Killing |
Stopping container kube-rbac-proxy | |
| (x2) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-87be3-predictor-845b844549-cpcg2 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-87be3-predictor-serving-cert" not found |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-87be3-predictor-5656fc75d4 |
SuccessfulCreate |
Created pod: error-404-isvc-87be3-predictor-5656fc75d4-x59s5 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-87be3-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-87be3-predictor-5656fc75d4 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-87be3-predictor-845b844549 |
SuccessfulCreate |
Created pod: success-200-isvc-87be3-predictor-845b844549-cpcg2 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-87be3-predictor-5656fc75d4-x59s5 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-87be3-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-87be3-predictor-845b844549 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-87be3-predictor-5656fc75d4-x59s5 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-87be3-predictor-5656fc75d4-x59s5 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-87be3 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-87be3": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-87be3": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-87be3 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-87be3": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f794f-predictor-75b57f85bc-4s5nf |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f794f-predictor-f4899d6f4-94s6d |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f794f-predictor-f4899d6f4-94s6d |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-87be3-predictor-5656fc75d4-x59s5 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-87be3-predictor-5656fc75d4-x59s5 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-87be3-predictor-5656fc75d4-x59s5 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1946" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-87be3-predictor-5656fc75d4-x59s5 |
AddedInterface |
Add eth0 [10.134.0.36/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
success-200-isvc-87be3-predictor-845b844549-cpcg2 |
AddedInterface |
Add eth0 [10.133.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-87be3-predictor-845b844549-cpcg2 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1946" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-87be3-predictor-845b844549-cpcg2 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-87be3-predictor-845b844549-cpcg2 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
InferenceServiceReady |
InferenceService [isvc-xgboost-graph] is Ready | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-87be3-predictor-845b844549-cpcg2 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-87be3-predictor-845b844549-cpcg2 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-87be3-predictor-845b844549-cpcg2 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-1] is Ready | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-2] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
model-chainer |
ScalingReplicaSet |
Scaled up replica set model-chainer-546cb8556c from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
model-chainer |
InferenceGraphReady |
InferenceGraph [model-chainer] is Ready | |
kserve-ci-e2e-test |
kubelet |
model-chainer-546cb8556c-kwp6s |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "model-chainer-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
model-chainer-546cb8556c |
SuccessfulCreate |
Created pod: model-chainer-546cb8556c-kwp6s | |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
multus |
model-chainer-546cb8556c-kwp6s |
AddedInterface |
Add eth0 [10.132.0.18/23] from ovn-kubernetes | |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
model-chainer-546cb8556c-kwp6s |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:2ae619f4eea9fa406199e7b8e562bc31c295665fd082c5f377ac0602bcc5b2e9" already present on machine | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
model-chainer-546cb8556c-kwp6s |
Started |
Started container model-chainer | |
kserve-ci-e2e-test |
kubelet |
model-chainer-546cb8556c-kwp6s |
Created |
Created container: model-chainer | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-f794f-6766495c86-m57t4 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-87be3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-87be3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-87be3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-87be3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7f476ffc7-bw8k9 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-766ff5986c-vkntk |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7f476ffc7-bw8k9 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-766ff5986c-vkntk |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
model-chainer-546cb8556c-kwp6s |
Killing |
Stopping container model-chainer | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-30f77-predictor-6b79cc6889 |
SuccessfulCreate |
Created pod: success-200-isvc-30f77-predictor-6b79cc6889-k52x5 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-30f77-predictor-6c9bbd784d |
SuccessfulCreate |
Created pod: error-404-isvc-30f77-predictor-6c9bbd784d-pwj9w | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-d85h2 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-30f77-predictor-6c9bbd784d-pwj9w |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-30f77-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-30f77-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-30f77-predictor-6c9bbd784d from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-30f77-predictor-6b79cc6889-k52x5 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-30f77-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-30f77-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-30f77-predictor-6b79cc6889 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-d85h2 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-30f77-predictor-6c9bbd784d-pwj9w |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-30f77-predictor-6b79cc6889-k52x5 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1946" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-30f77-predictor-6c9bbd784d-pwj9w |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-766ff5986c-vkntk |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.16:8643/healthz": dial tcp 10.132.0.16:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-30f77-predictor-6c9bbd784d-pwj9w |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-30f77-predictor-6c9bbd784d-pwj9w |
Created |
Created container: kserve-container | |
| (x10) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-766ff5986c-vkntk |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.16:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-30f77-predictor-6c9bbd784d-pwj9w |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1946" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-d85h2 |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.26:8643/healthz": dial tcp 10.133.0.26:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-30f77-predictor-6b79cc6889-k52x5 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-30f77-predictor-6b79cc6889-k52x5 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-30f77-predictor-6c9bbd784d-pwj9w |
AddedInterface |
Add eth0 [10.134.0.37/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
success-200-isvc-30f77-predictor-6b79cc6889-k52x5 |
AddedInterface |
Add eth0 [10.133.0.28/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-30f77-predictor-6c9bbd784d-pwj9w |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-30f77-predictor-6b79cc6889-k52x5 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-30f77-predictor-6b79cc6889-k52x5 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-30f77-predictor-6b79cc6889-k52x5 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-7f476ffc7-bw8k9 |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.25:8643/healthz": dial tcp 10.133.0.25:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-87be3-predictor-845b844549-cpcg2 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.27:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-87be3 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-87be3 |
InferenceServiceReady |
InferenceService [error-404-isvc-87be3] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-87be3 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-87be3 |
InferenceServiceReady |
InferenceService [success-200-isvc-87be3] is Ready | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-30f77-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-30f77-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
kubelet |
switch-graph-87be3-bdb646768-z4lgk |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:2ae619f4eea9fa406199e7b8e562bc31c295665fd082c5f377ac0602bcc5b2e9" already present on machine | |
kserve-ci-e2e-test |
kubelet |
switch-graph-87be3-bdb646768-z4lgk |
Started |
Started container switch-graph-87be3 | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-87be3-bdb646768 |
SuccessfulCreate |
Created pod: switch-graph-87be3-bdb646768-z4lgk | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-87be3 |
UpdateFailed |
Failed to update status for InferenceGraph "switch-graph-87be3": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "switch-graph-87be3": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-87be3 |
InferenceGraphReady |
InferenceGraph [switch-graph-87be3] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-87be3 |
ScalingReplicaSet |
Scaled up replica set switch-graph-87be3-bdb646768 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
switch-graph-87be3-bdb646768-z4lgk |
AddedInterface |
Add eth0 [10.132.0.19/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-87be3-bdb646768-z4lgk |
Created |
Created container: switch-graph-87be3 | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-87be3 |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "switch-graph-87be3": the object has been modified; please apply your changes to the latest version and try again | |
| (x6) | kserve-ci-e2e-test |
kubelet |
model-chainer-546cb8556c-kwp6s |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-30f77-predictor-6b79cc6889-k52x5 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.28:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-87be3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-87be3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-30f77-predictor-6c9bbd784d-pwj9w |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.37:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-87be3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-87be3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-30f77-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-30f77-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-30f77 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-30f77 |
InferenceServiceReady |
InferenceService [success-200-isvc-30f77] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-30f77 |
InferenceServiceReady |
InferenceService [error-404-isvc-30f77] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-30f77 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-87be3 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-87be3 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-30f77 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-30f77-859fc5c779 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-30f77-859fc5c779 |
SuccessfulCreate |
Created pod: sequence-graph-30f77-859fc5c779-w9th4 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-30f77-859fc5c779-w9th4 |
Started |
Started container sequence-graph-30f77 | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-30f77 |
InferenceGraphReady |
InferenceGraph [sequence-graph-30f77] is Ready | |
kserve-ci-e2e-test |
multus |
sequence-graph-30f77-859fc5c779-w9th4 |
AddedInterface |
Add eth0 [10.132.0.20/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-30f77-859fc5c779-w9th4 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:2ae619f4eea9fa406199e7b8e562bc31c295665fd082c5f377ac0602bcc5b2e9" already present on machine | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-30f77-859fc5c779-w9th4 |
Created |
Created container: sequence-graph-30f77 | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-30f77-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-30f77-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-30f77-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-30f77-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-30f77 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-30f77 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
switch-graph-87be3-bdb646768-z4lgk |
Killing |
Stopping container switch-graph-87be3 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-601fd-predictor-5978f84496-s6xpw |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-601fd-predictor-5978f84496-s6xpw |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1946" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-87be3-predictor-845b844549-cpcg2 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-87be3-predictor-845b844549-cpcg2 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-601fd-predictor-5978f84496 |
SuccessfulCreate |
Created pod: error-404-isvc-601fd-predictor-5978f84496-s6xpw | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-87be3-predictor-5656fc75d4-x59s5 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-601fd-predictor-5978f84496-s6xpw |
AddedInterface |
Add eth0 [10.134.0.38/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-601fd-predictor-5978f84496-s6xpw |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-601fd-predictor-646cfc64df-ckjl2 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-601fd-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-87be3-predictor-5656fc75d4-x59s5 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-601fd-predictor-5978f84496-s6xpw |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-601fd-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-601fd-predictor-646cfc64df from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-601fd-predictor-646cfc64df |
SuccessfulCreate |
Created pod: success-200-isvc-601fd-predictor-646cfc64df-ckjl2 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-601fd-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-601fd-predictor-5978f84496 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-601fd-predictor-5978f84496-s6xpw |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-601fd-predictor-646cfc64df-ckjl2 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-601fd-predictor-646cfc64df-ckjl2 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-601fd-predictor-646cfc64df-ckjl2 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1946" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-601fd-predictor-646cfc64df-ckjl2 |
AddedInterface |
Add eth0 [10.133.0.29/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-601fd-predictor-646cfc64df-ckjl2 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-601fd-predictor-646cfc64df-ckjl2 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-601fd-predictor-646cfc64df-ckjl2 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-601fd-predictor-5978f84496-s6xpw |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-87be3-predictor-5656fc75d4-x59s5 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.36:8643/healthz": dial tcp 10.134.0.36:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-87be3-predictor-5656fc75d4-x59s5 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.36:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-87be3-bdb646768-z4lgk |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-601fd-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-601fd-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-601fd-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-601fd-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
sequence-graph-30f77-859fc5c779-w9th4 |
Killing |
Stopping container sequence-graph-30f77 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-30f77-predictor-6b79cc6889-k52x5 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-30f77-predictor-6c9bbd784d-pwj9w |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-30f77-predictor-6c9bbd784d-pwj9w |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-30f77-predictor-6b79cc6889-k52x5 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-e90df-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-e90df-predictor-5bbcb468bc from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-e90df-predictor-5bbcb468bc |
SuccessfulCreate |
Created pod: success-200-isvc-e90df-predictor-5bbcb468bc-wkj7z | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-e90df-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-e90df-predictor-5b7bc4c674 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e90df-predictor-5b7bc4c674-96vrn |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e90df-predictor-5b7bc4c674-96vrn |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e90df-predictor-5b7bc4c674-96vrn |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e90df-predictor-5b7bc4c674-96vrn |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e90df-predictor-5b7bc4c674-96vrn |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e90df-predictor-5b7bc4c674-96vrn |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1946" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-e90df-predictor-5b7bc4c674-96vrn |
AddedInterface |
Add eth0 [10.134.0.39/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-e90df-predictor-5b7bc4c674 |
SuccessfulCreate |
Created pod: error-404-isvc-e90df-predictor-5b7bc4c674-96vrn | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e90df-predictor-5bbcb468bc-wkj7z |
FailedMount |
MountVolume.SetUp failed for volume "kube-rbac-proxy-sar-config" : failed to sync configmap cache: timed out waiting for the condition | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e90df-predictor-5bbcb468bc-wkj7z |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : failed to sync secret cache: timed out waiting for the condition | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e90df-predictor-5bbcb468bc-wkj7z |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e90df-predictor-5bbcb468bc-wkj7z |
Started |
Started container kube-rbac-proxy | |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-601fd-predictor-5978f84496-s6xpw |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.38:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e90df-predictor-5bbcb468bc-wkj7z |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e90df-predictor-5bbcb468bc-wkj7z |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e90df-predictor-5bbcb468bc-wkj7z |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-e90df-predictor-5bbcb468bc-wkj7z |
AddedInterface |
Add eth0 [10.133.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e90df-predictor-5bbcb468bc-wkj7z |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1946" already present on machine | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-601fd-predictor-646cfc64df-ckjl2 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.29:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-30f77-predictor-6b79cc6889-k52x5 |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.28:8643/healthz": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-30f77-predictor-6b79cc6889-k52x5 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.28:8080: i/o timeout | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-601fd |
InferenceServiceReady |
InferenceService [error-404-isvc-601fd] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-601fd |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-601fd |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-601fd |
InferenceServiceReady |
InferenceService [success-200-isvc-601fd] is Ready | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-601fd-8f5c764c5-5vlqq |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "ensemble-graph-601fd-serving-cert" not found | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-601fd-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-601fd-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-601fd |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-601fd-8f5c764c5 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-601fd-8f5c764c5 |
SuccessfulCreate |
Created pod: ensemble-graph-601fd-8f5c764c5-5vlqq | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-601fd-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-601fd-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-601fd |
InferenceGraphReady |
InferenceGraph [ensemble-graph-601fd] is Ready | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-601fd-8f5c764c5-5vlqq |
Created |
Created container: ensemble-graph-601fd | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-601fd-8f5c764c5-5vlqq |
Started |
Started container ensemble-graph-601fd | |
kserve-ci-e2e-test |
multus |
ensemble-graph-601fd-8f5c764c5-5vlqq |
AddedInterface |
Add eth0 [10.132.0.21/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-601fd-8f5c764c5-5vlqq |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:2ae619f4eea9fa406199e7b8e562bc31c295665fd082c5f377ac0602bcc5b2e9" already present on machine | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-30f77-859fc5c779-w9th4 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-601fd-predictor-646cfc64df-ckjl2 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-64898-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-64898-predictor-57884f876f from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-601fd-predictor-5978f84496-s6xpw |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-601fd-predictor-646cfc64df-ckjl2 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-601fd-predictor-5978f84496-s6xpw |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-601fd-8f5c764c5-5vlqq |
Killing |
Stopping container ensemble-graph-601fd | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-64898-predictor-57884f876f-8wq7r |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-64898-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-64898-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-64898-predictor-c55974db4 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-64898-predictor-c55974db4 |
SuccessfulCreate |
Created pod: error-404-isvc-64898-predictor-c55974db4-6czmd | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-64898-predictor-57884f876f |
SuccessfulCreate |
Created pod: success-200-isvc-64898-predictor-57884f876f-8wq7r | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-64898-predictor-c55974db4-6czmd |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1946" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-64898-predictor-57884f876f-8wq7r |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-64898-predictor-c55974db4-6czmd |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-64898-predictor-57884f876f-8wq7r |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-64898-predictor-c55974db4-6czmd |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-64898-predictor-c55974db4-6czmd |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-64898-predictor-c55974db4-6czmd |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-64898-predictor-c55974db4-6czmd |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-64898-predictor-57884f876f-8wq7r |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-64898-predictor-c55974db4-6czmd |
AddedInterface |
Add eth0 [10.134.0.40/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
success-200-isvc-64898-predictor-57884f876f-8wq7r |
AddedInterface |
Add eth0 [10.133.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-64898-predictor-57884f876f-8wq7r |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1946" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-64898-predictor-57884f876f-8wq7r |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-64898-predictor-57884f876f-8wq7r |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-e90df-predictor-5b7bc4c674-96vrn |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.39:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-e90df-predictor-5bbcb468bc-wkj7z |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.30:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e90df-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e90df-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e90df-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e90df-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e90df |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e90df |
InferenceServiceReady |
InferenceService [error-404-isvc-e90df] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e90df |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e90df |
InferenceServiceReady |
InferenceService [success-200-isvc-e90df] is Ready | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-e90df |
InferenceGraphReady |
InferenceGraph [sequence-graph-e90df] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-e90df |
ScalingReplicaSet |
Scaled up replica set sequence-graph-e90df-764845479d from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-e90df-764845479d |
SuccessfulCreate |
Created pod: sequence-graph-e90df-764845479d-h54j2 | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e90df-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-e90df |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "sequence-graph-e90df": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-e90df |
UpdateFailed |
Failed to update status for InferenceGraph "sequence-graph-e90df": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "sequence-graph-e90df": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e90df-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e90df-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e90df-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-e90df-764845479d-h54j2 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "sequence-graph-e90df-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-e90df-764845479d-h54j2 |
Created |
Created container: sequence-graph-e90df | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-e90df-764845479d-h54j2 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:2ae619f4eea9fa406199e7b8e562bc31c295665fd082c5f377ac0602bcc5b2e9" already present on machine | |
kserve-ci-e2e-test |
multus |
sequence-graph-e90df-764845479d-h54j2 |
AddedInterface |
Add eth0 [10.132.0.22/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-e90df-764845479d-h54j2 |
Started |
Started container sequence-graph-e90df | |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-601fd-8f5c764c5-5vlqq |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
sequence-graph-e90df-764845479d-h54j2 |
Killing |
Stopping container sequence-graph-e90df | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-c7da0-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-c7da0-predictor-5488f9f968 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c7da0-predictor-945f88485-8srks |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e90df-predictor-5bbcb468bc-wkj7z |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c7da0-predictor-945f88485-8srks |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c7da0-predictor-945f88485-8srks |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e90df-predictor-5bbcb468bc-wkj7z |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c7da0-predictor-945f88485-8srks |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1946" already present on machine | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-c7da0-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-c7da0-predictor-945f88485 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
success-200-isvc-c7da0-predictor-945f88485-8srks |
AddedInterface |
Add eth0 [10.133.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e90df-predictor-5b7bc4c674-96vrn |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e90df-predictor-5b7bc4c674-96vrn |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-c7da0-predictor-5488f9f968 |
SuccessfulCreate |
Created pod: error-404-isvc-c7da0-predictor-5488f9f968-qtw88 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-c7da0-predictor-945f88485 |
SuccessfulCreate |
Created pod: success-200-isvc-c7da0-predictor-945f88485-8srks | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c7da0-predictor-5488f9f968-qtw88 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-c7da0-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c7da0-predictor-5488f9f968-qtw88 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c7da0-predictor-5488f9f968-qtw88 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-c7da0-predictor-5488f9f968-qtw88 |
AddedInterface |
Add eth0 [10.134.0.41/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c7da0-predictor-5488f9f968-qtw88 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1946" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c7da0-predictor-945f88485-8srks |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c7da0-predictor-5488f9f968-qtw88 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c7da0-predictor-5488f9f968-qtw88 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c7da0-predictor-5488f9f968-qtw88 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c7da0-predictor-945f88485-8srks |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e90df-predictor-5b7bc4c674-96vrn |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.39:8643/healthz": dial tcp 10.134.0.39:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-64898-predictor-57884f876f-8wq7r |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.31:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-64898-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-64898-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-64898 |
InferenceServiceReady |
InferenceService [error-404-isvc-64898] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-64898 |
InferenceServiceReady |
InferenceService [success-200-isvc-64898] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-64898 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-64898 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c7da0-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c7da0-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-64898 |
InferenceGraphReady |
InferenceGraph [ensemble-graph-64898] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-64898 |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-64898-76dcdb6787 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-64898-76dcdb6787 |
SuccessfulCreate |
Created pod: ensemble-graph-64898-76dcdb6787-r4g2b | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-64898-76dcdb6787-r4g2b |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "ensemble-graph-64898-serving-cert" not found | |
kserve-ci-e2e-test |
multus |
ensemble-graph-64898-76dcdb6787-r4g2b |
AddedInterface |
Add eth0 [10.132.0.23/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-64898-76dcdb6787-r4g2b |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:2ae619f4eea9fa406199e7b8e562bc31c295665fd082c5f377ac0602bcc5b2e9" already present on machine | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-64898-76dcdb6787-r4g2b |
Started |
Started container ensemble-graph-64898 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-64898-76dcdb6787-r4g2b |
Created |
Created container: ensemble-graph-64898 | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-e90df-764845479d-h54j2 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c7da0-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c7da0-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-c7da0-predictor-5488f9f968-qtw88 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.41:8080: connect: connection refused |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-64898-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-64898-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-64898-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-64898-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-c7da0 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-c7da0 |
InferenceServiceReady |
InferenceService [success-200-isvc-c7da0] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-c7da0 |
InferenceServiceReady |
InferenceService [error-404-isvc-c7da0] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-c7da0 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-64898 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-64898 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-c7da0-7b7778f46 |
SuccessfulCreate |
Created pod: sequence-graph-c7da0-7b7778f46-tqcxt | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-c7da0 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-c7da0-7b7778f46 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-c7da0 |
InferenceGraphReady |
InferenceGraph [sequence-graph-c7da0] is Ready | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-c7da0-7b7778f46-tqcxt |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "sequence-graph-c7da0-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-c7da0-7b7778f46-tqcxt |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:2ae619f4eea9fa406199e7b8e562bc31c295665fd082c5f377ac0602bcc5b2e9" already present on machine | |
kserve-ci-e2e-test |
multus |
sequence-graph-c7da0-7b7778f46-tqcxt |
AddedInterface |
Add eth0 [10.132.0.24/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-c7da0-7b7778f46-tqcxt |
Started |
Started container sequence-graph-c7da0 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-c7da0-7b7778f46-tqcxt |
Created |
Created container: sequence-graph-c7da0 | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c7da0-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c7da0-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c7da0-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c7da0-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-c7da0 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-c7da0 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-64898-predictor-c55974db4-6czmd |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-64898-76dcdb6787-r4g2b |
Killing |
Stopping container ensemble-graph-64898 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-64898-predictor-57884f876f-8wq7r |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-64898-predictor-57884f876f-8wq7r |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-13b40 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-13b40": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-13b40 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-13b40": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-13b40 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-13b40": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-13b40": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
multus |
success-200-isvc-13b40-predictor-8bb687fb7-g48nb |
AddedInterface |
Add eth0 [10.133.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-13b40-predictor-5cf559cf78 |
SuccessfulCreate |
Created pod: error-404-isvc-13b40-predictor-5cf559cf78-8pgvv | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-13b40-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-13b40-predictor-5cf559cf78 from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-13b40-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-13b40-predictor-8bb687fb7 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-13b40-predictor-8bb687fb7 |
SuccessfulCreate |
Created pod: success-200-isvc-13b40-predictor-8bb687fb7-g48nb | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-64898-predictor-c55974db4-6czmd |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-13b40 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-13b40": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-13b40": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-13b40-predictor-5cf559cf78-8pgvv |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1946" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-13b40-predictor-5cf559cf78-8pgvv |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-13b40-predictor-5cf559cf78-8pgvv |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-13b40-predictor-8bb687fb7-g48nb |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-13b40-predictor-8bb687fb7-g48nb |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-13b40-predictor-8bb687fb7-g48nb |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-13b40-predictor-5cf559cf78-8pgvv |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-13b40-predictor-5cf559cf78-8pgvv |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-13b40-predictor-8bb687fb7-g48nb |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-13b40-predictor-8bb687fb7-g48nb |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-13b40-predictor-5cf559cf78-8pgvv |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-13b40-predictor-8bb687fb7-g48nb |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1946" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-13b40-predictor-5cf559cf78-8pgvv |
AddedInterface |
Add eth0 [10.134.0.42/23] from ovn-kubernetes | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-64898-predictor-c55974db4-6czmd |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.40:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-64898-predictor-c55974db4-6czmd |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.40:8643/healthz": dial tcp 10.134.0.40:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-64898-76dcdb6787-r4g2b |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-13b40-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-13b40-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-13b40-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-13b40-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c7da0-predictor-945f88485-8srks |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c7da0-predictor-945f88485-8srks |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-ce1f6-predictor-76644689b9 |
SuccessfulCreate |
Created pod: success-200-isvc-ce1f6-predictor-76644689b9-smjt7 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c7da0-predictor-5488f9f968-qtw88 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c7da0-predictor-5488f9f968-qtw88 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-c7da0-7b7778f46-tqcxt |
Killing |
Stopping container sequence-graph-c7da0 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-ce1f6 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-ce1f6": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-ce1f6 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-ce1f6": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-ce1f6": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-ce1f6-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-ce1f6-predictor-76644689b9 from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-ce1f6 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-ce1f6": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-ce1f6 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-ce1f6": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-ce1f6": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ce1f6-predictor-76644689b9-smjt7 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-ce1f6-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ce1f6-predictor-b77d84c64-v2lwk |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-ce1f6-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-ce1f6-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-ce1f6-predictor-b77d84c64 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-ce1f6-predictor-b77d84c64 |
SuccessfulCreate |
Created pod: error-404-isvc-ce1f6-predictor-b77d84c64-v2lwk | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ce1f6-predictor-76644689b9-smjt7 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ce1f6-predictor-b77d84c64-v2lwk |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ce1f6-predictor-b77d84c64-v2lwk |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ce1f6-predictor-76644689b9-smjt7 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ce1f6-predictor-b77d84c64-v2lwk |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ce1f6-predictor-76644689b9-smjt7 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ce1f6-predictor-b77d84c64-v2lwk |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ce1f6-predictor-76644689b9-smjt7 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ce1f6-predictor-b77d84c64-v2lwk |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ce1f6-predictor-76644689b9-smjt7 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ce1f6-predictor-76644689b9-smjt7 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1946" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-ce1f6-predictor-76644689b9-smjt7 |
AddedInterface |
Add eth0 [10.133.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ce1f6-predictor-b77d84c64-v2lwk |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1946" already present on machine | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-13b40-predictor-8bb687fb7-g48nb |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.33:8080: connect: connection refused |
kserve-ci-e2e-test |
multus |
error-404-isvc-ce1f6-predictor-b77d84c64-v2lwk |
AddedInterface |
Add eth0 [10.134.0.43/23] from ovn-kubernetes | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-c7da0-predictor-945f88485-8srks |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.32:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-13b40-predictor-5cf559cf78-8pgvv |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.42:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c7da0-predictor-945f88485-8srks |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.32:8643/healthz": dial tcp 10.133.0.32:8643: connect: connection refused | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-13b40 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-13b40 |
InferenceServiceReady |
InferenceService [error-404-isvc-13b40] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-13b40 |
InferenceServiceReady |
InferenceService [success-200-isvc-13b40] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-13b40 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-ce1f6-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-ce1f6-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-ce1f6-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-ce1f6-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-13b40-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-13b40-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
splitter-graph-13b40-64fb84556f-s67d4 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "splitter-graph-13b40-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-13b40-64fb84556f |
SuccessfulCreate |
Created pod: splitter-graph-13b40-64fb84556f-s67d4 | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-13b40 |
ScalingReplicaSet |
Scaled up replica set splitter-graph-13b40-64fb84556f from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-13b40 |
InferenceGraphReady |
InferenceGraph [splitter-graph-13b40] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-13b40-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-13b40-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
splitter-graph-13b40-64fb84556f-s67d4 |
Started |
Started container splitter-graph-13b40 | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-13b40-64fb84556f-s67d4 |
Created |
Created container: splitter-graph-13b40 | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-13b40-64fb84556f-s67d4 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:2ae619f4eea9fa406199e7b8e562bc31c295665fd082c5f377ac0602bcc5b2e9" already present on machine | |
kserve-ci-e2e-test |
multus |
splitter-graph-13b40-64fb84556f-s67d4 |
AddedInterface |
Add eth0 [10.132.0.25/23] from ovn-kubernetes | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-c7da0-7b7778f46-tqcxt |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-13b40-predictor-5cf559cf78-8pgvv |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-13b40-64fb84556f-s67d4 |
Killing |
Stopping container splitter-graph-13b40 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-7d014-predictor-57c444947d |
SuccessfulCreate |
Created pod: error-404-isvc-7d014-predictor-57c444947d-llcg9 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-7d014-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-7d014-predictor-57c444947d from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-13b40-predictor-5cf559cf78-8pgvv |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-7d014-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-7d014-predictor-5688f58dbc from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-7d014-predictor-5688f58dbc |
SuccessfulCreate |
Created pod: success-200-isvc-7d014-predictor-5688f58dbc-7pbff | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-13b40-predictor-8bb687fb7-g48nb |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-13b40-predictor-8bb687fb7-g48nb |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
multus |
success-200-isvc-7d014-predictor-5688f58dbc-7pbff |
AddedInterface |
Add eth0 [10.133.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-7d014-predictor-57c444947d-llcg9 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-7d014-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-7d014-predictor-5688f58dbc-7pbff |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-7d014-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-7d014-predictor-5688f58dbc-7pbff |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-7d014-predictor-57c444947d-llcg9 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-13b40-predictor-8bb687fb7-g48nb |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.33:8643/healthz": dial tcp 10.133.0.33:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-7d014-predictor-5688f58dbc-7pbff |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1946" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-7d014-predictor-57c444947d-llcg9 |
AddedInterface |
Add eth0 [10.134.0.44/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-7d014-predictor-5688f58dbc-7pbff |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-7d014-predictor-5688f58dbc-7pbff |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-7d014-predictor-57c444947d-llcg9 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1946" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-7d014-predictor-5688f58dbc-7pbff |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-7d014-predictor-57c444947d-llcg9 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-7d014-predictor-5688f58dbc-7pbff |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-7d014-predictor-57c444947d-llcg9 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-7d014-predictor-57c444947d-llcg9 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-7d014-predictor-57c444947d-llcg9 |
Started |
Started container kserve-container | |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-ce1f6-predictor-b77d84c64-v2lwk |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.43:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-13b40-predictor-5cf559cf78-8pgvv |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.42:8643/healthz": dial tcp 10.134.0.42:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-ce1f6-predictor-76644689b9-smjt7 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.34:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-ce1f6 |
InferenceServiceReady |
InferenceService [error-404-isvc-ce1f6] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-ce1f6 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-ce1f6 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-ce1f6 |
InferenceServiceReady |
InferenceService [success-200-isvc-ce1f6] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-ce1f6-75dbfb64bd |
SuccessfulCreate |
Created pod: switch-graph-ce1f6-75dbfb64bd-q4pxj | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-ce1f6 |
ScalingReplicaSet |
Scaled up replica set switch-graph-ce1f6-75dbfb64bd from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-ce1f6-75dbfb64bd-q4pxj |
Created |
Created container: switch-graph-ce1f6 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-ce1f6-75dbfb64bd-q4pxj |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:2ae619f4eea9fa406199e7b8e562bc31c295665fd082c5f377ac0602bcc5b2e9" already present on machine | |
kserve-ci-e2e-test |
multus |
switch-graph-ce1f6-75dbfb64bd-q4pxj |
AddedInterface |
Add eth0 [10.132.0.26/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-ce1f6-75dbfb64bd-q4pxj |
Started |
Started container switch-graph-ce1f6 | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-ce1f6 |
InferenceGraphReady |
InferenceGraph [switch-graph-ce1f6] is Ready | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-ce1f6-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-ce1f6-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-13b40-64fb84556f-s67d4 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-7d014-predictor-57c444947d-llcg9 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.44:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-7d014-predictor-5688f58dbc-7pbff |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.35:8080: connect: connection refused |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-ce1f6 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-ce1f6-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-ce1f6-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-ce1f6 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-7d014-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-7d014-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-7d014 |
InferenceServiceReady |
InferenceService [error-404-isvc-7d014] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-7d014 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-7d014 |
InferenceServiceReady |
InferenceService [success-200-isvc-7d014] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-7d014 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-ce1f6 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-ce1f6 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-7d014-7f8d646646 |
SuccessfulCreate |
Created pod: splitter-graph-7d014-7f8d646646-jnp9w | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-7d014 |
ScalingReplicaSet |
Scaled up replica set splitter-graph-7d014-7f8d646646 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-7d014 |
InferenceGraphReady |
InferenceGraph [splitter-graph-7d014] is Ready | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-7d014-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-7d014-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
splitter-graph-7d014-7f8d646646-jnp9w |
Created |
Created container: splitter-graph-7d014 | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-7d014-7f8d646646-jnp9w |
Started |
Started container splitter-graph-7d014 | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-7d014-7f8d646646-jnp9w |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:2ae619f4eea9fa406199e7b8e562bc31c295665fd082c5f377ac0602bcc5b2e9" already present on machine | |
kserve-ci-e2e-test |
multus |
splitter-graph-7d014-7f8d646646-jnp9w |
AddedInterface |
Add eth0 [10.132.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-7d014 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-7d014-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-7d014-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-7d014-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-7d014 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-7d014-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-7d014 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-7d014 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-7d014-7f8d646646-jnp9w |
Killing |
Stopping container splitter-graph-7d014 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-7d014-predictor-5688f58dbc-7pbff |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-7d014-predictor-57c444947d-llcg9 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-7d014-predictor-57c444947d-llcg9 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-7d014-predictor-5688f58dbc-7pbff |
Killing |
Stopping container kube-rbac-proxy | |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-7d014-7f8d646646-jnp9w |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ce1f6-predictor-76644689b9-smjt7 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ce1f6-predictor-b77d84c64-v2lwk |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ce1f6-predictor-b77d84c64-v2lwk |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
switch-graph-ce1f6-75dbfb64bd-q4pxj |
Killing |
Stopping container switch-graph-ce1f6 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ce1f6-predictor-76644689b9-smjt7 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ce1f6-predictor-76644689b9-smjt7 |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.34:8643/healthz": dial tcp 10.133.0.34:8643: connect: connection refused | |
| (x5) | kserve-ci-e2e-test |
kubelet |
switch-graph-ce1f6-75dbfb64bd-q4pxj |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |