| Time | Namespace | Component | RelatedObject | Reason | Message |
|---|---|---|---|---|---|
kserve-ci-e2e-test |
success-200-isvc-f8431-predictor-6bcffc9d57-nq2nz |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-f8431-predictor-6bcffc9d57-nq2nz to ip-10-0-142-202.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-4aba3-5f6b4f4ff7-vzzq7 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-4aba3-5f6b4f4ff7-vzzq7 to ip-10-0-133-161.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-2e6fe-dcb8bc6f8-mvk48 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-2e6fe-dcb8bc6f8-mvk48 to ip-10-0-133-161.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-2e6fe-predictor-89f79697d-42755 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-2e6fe-predictor-89f79697d-42755 to ip-10-0-132-138.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-5c921-predictor-8548d846d5-287kf |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-5c921-predictor-8548d846d5-287kf to ip-10-0-142-202.ec2.internal | ||
kserve-ci-e2e-test |
isvc-xgboost-graph-predictor-669d8d6456-p5mg2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-xgboost-graph-predictor-669d8d6456-p5mg2 to ip-10-0-142-202.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-e938d-798f566f4f-4lmhz |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-e938d-798f566f4f-4lmhz to ip-10-0-133-161.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-5c921-predictor-78cfcc75d6-6fxpn |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-5c921-predictor-78cfcc75d6-6fxpn to ip-10-0-132-138.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-4aba3-predictor-6dcd794499-dzrk9 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-4aba3-predictor-6dcd794499-dzrk9 to ip-10-0-142-202.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-be7a5-predictor-6654cff85-qtt26 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-be7a5-predictor-6654cff85-qtt26 to ip-10-0-142-202.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-d0532-predictor-6cfcdcc985-fjlp5 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-d0532-predictor-6cfcdcc985-fjlp5 to ip-10-0-142-202.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-e938d-predictor-858f86b7bc-ckndq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-e938d-predictor-858f86b7bc-ckndq to ip-10-0-142-202.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-44b1e-predictor-589485df45-9jvsp |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-44b1e-predictor-589485df45-9jvsp to ip-10-0-142-202.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-5c921-c79bf4c85-pqgsq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-5c921-c79bf4c85-pqgsq to ip-10-0-133-161.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-4aba3-predictor-6f567c46fb-kdltw |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-4aba3-predictor-6f567c46fb-kdltw to ip-10-0-132-138.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-32c7c-predictor-57df59874f-7vtc8 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-32c7c-predictor-57df59874f-7vtc8 to ip-10-0-142-202.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-2e6fe-predictor-66c4d45c7c-xmb5t |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-2e6fe-predictor-66c4d45c7c-xmb5t to ip-10-0-142-202.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-d0532-predictor-7bbf598d6b-xswpz |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-d0532-predictor-7bbf598d6b-xswpz to ip-10-0-132-138.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-d0532-75644c6c8f-ccgwj |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-d0532-75644c6c8f-ccgwj to ip-10-0-133-161.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-1b9dc-5995d747dd-q5dbh |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-1b9dc-5995d747dd-q5dbh to ip-10-0-133-161.ec2.internal | ||
kserve-ci-e2e-test |
model-chainer-85d456f99f-98r87 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/model-chainer-85d456f99f-98r87 to ip-10-0-133-161.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-f8431-7f98d55687-f4dl5 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-f8431-7f98d55687-f4dl5 to ip-10-0-133-161.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-be7a5-predictor-5c7766ddbb-8xswq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-be7a5-predictor-5c7766ddbb-8xswq to ip-10-0-132-138.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-1b9dc-predictor-7c988b6956-f7lkv |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-1b9dc-predictor-7c988b6956-f7lkv to ip-10-0-132-138.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-44b1e-predictor-648b8cfbf8-mdq6n |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-44b1e-predictor-648b8cfbf8-mdq6n to ip-10-0-132-138.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-1b9dc-predictor-68868569ff-tdd6b |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-1b9dc-predictor-68868569ff-tdd6b to ip-10-0-142-202.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-be7a5-64dbc87fdf-ddr2h |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-be7a5-64dbc87fdf-ddr2h to ip-10-0-133-161.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-44b1e-98dcdcd84-rr45w |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-44b1e-98dcdcd84-rr45w to ip-10-0-133-161.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-e938d-predictor-8478f5b96-brdgr |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-e938d-predictor-8478f5b96-brdgr to ip-10-0-132-138.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-f8431-predictor-6b847f4b94-q6dxt |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-f8431-predictor-6b847f4b94-q6dxt to ip-10-0-132-138.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-1-predictor-56dcdd5cdf-fdd6m |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-56dcdd5cdf-fdd6m to ip-10-0-133-161.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-32c7c-6ff7f8b44c-rwdm4 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-32c7c-6ff7f8b44c-rwdm4 to ip-10-0-133-161.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-2-predictor-7db9dc6879-jr686 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7db9dc6879-jr686 to ip-10-0-132-138.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-32c7c-predictor-69fdd96fd6-v6c7z |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-32c7c-predictor-69fdd96fd6-v6c7z to ip-10-0-132-138.ec2.internal | ||
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-1-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-1-predictor-56dcdd5cdf from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
UpdateFailed |
Failed to update status for InferenceService "isvc-sklearn-graph-1": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "isvc-sklearn-graph-1": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "isvc-sklearn-graph-1": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-1-predictor-56dcdd5cdf |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-1-predictor-56dcdd5cdf-fdd6m | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-1b9dc-predictor-68868569ff |
SuccessfulCreate |
Created pod: success-200-isvc-1b9dc-predictor-68868569ff-tdd6b | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-1b9dc-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-1b9dc-predictor-68868569ff from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
isvc-xgboost-graph-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-xgboost-graph-predictor-669d8d6456 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b9dc-predictor-68868569ff-tdd6b |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-1b9dc-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-2-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-2-predictor-7db9dc6879 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-xgboost-graph-predictor-669d8d6456 |
SuccessfulCreate |
Created pod: isvc-xgboost-graph-predictor-669d8d6456-p5mg2 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-1b9dc-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-1b9dc-predictor-7c988b6956 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b9dc-predictor-68868569ff-tdd6b |
Pulling |
Pulling image "quay.io/opendatahub/success-200-isvc:odh-pr-1443" | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-1b9dc-predictor-7c988b6956 |
SuccessfulCreate |
Created pod: error-404-isvc-1b9dc-predictor-7c988b6956-f7lkv | |
kserve-ci-e2e-test |
multus |
success-200-isvc-1b9dc-predictor-68868569ff-tdd6b |
AddedInterface |
Add eth0 [10.132.0.17/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-p5mg2 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "isvc-xgboost-graph-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
multus |
error-404-isvc-1b9dc-predictor-7c988b6956-f7lkv |
AddedInterface |
Add eth0 [10.134.0.29/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b9dc-predictor-7c988b6956-f7lkv |
Pulling |
Pulling image "quay.io/opendatahub/error-404-isvc:odh-pr-1443" | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-1-predictor-56dcdd5cdf-fdd6m |
AddedInterface |
Add eth0 [10.133.0.28/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-56dcdd5cdf-fdd6m |
Pulling |
Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:4ae4df75945dd799b0c11b5e37a48a5ba8230ff3507d345ea07c8897a8c39885" | |
kserve-ci-e2e-test |
multus |
isvc-xgboost-graph-predictor-669d8d6456-p5mg2 |
AddedInterface |
Add eth0 [10.132.0.18/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-2-predictor-7db9dc6879 |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-2-predictor-7db9dc6879-jr686 | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7db9dc6879-jr686 |
Pulling |
Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:4ae4df75945dd799b0c11b5e37a48a5ba8230ff3507d345ea07c8897a8c39885" | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-2-predictor-7db9dc6879-jr686 |
AddedInterface |
Add eth0 [10.134.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-p5mg2 |
Pulling |
Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:4ae4df75945dd799b0c11b5e37a48a5ba8230ff3507d345ea07c8897a8c39885" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-56dcdd5cdf-fdd6m |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-56dcdd5cdf-fdd6m |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:4ae4df75945dd799b0c11b5e37a48a5ba8230ff3507d345ea07c8897a8c39885" in 3.134s (3.134s including waiting). Image size: 301160875 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-56dcdd5cdf-fdd6m |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-56dcdd5cdf-fdd6m |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1443" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b9dc-predictor-7c988b6956-f7lkv |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7db9dc6879-jr686 |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:4ae4df75945dd799b0c11b5e37a48a5ba8230ff3507d345ea07c8897a8c39885" in 12.108s (12.108s including waiting). Image size: 301160875 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7db9dc6879-jr686 |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7db9dc6879-jr686 |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b9dc-predictor-7c988b6956-f7lkv |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b9dc-predictor-7c988b6956-f7lkv |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b9dc-predictor-7c988b6956-f7lkv |
Pulled |
Successfully pulled image "quay.io/opendatahub/error-404-isvc:odh-pr-1443" in 13.026s (13.026s including waiting). Image size: 1334679138 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b9dc-predictor-68868569ff-tdd6b |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-p5mg2 |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b9dc-predictor-68868569ff-tdd6b |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b9dc-predictor-68868569ff-tdd6b |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-p5mg2 |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:4ae4df75945dd799b0c11b5e37a48a5ba8230ff3507d345ea07c8897a8c39885" in 13.438s (13.438s including waiting). Image size: 301160875 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-p5mg2 |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b9dc-predictor-68868569ff-tdd6b |
Pulled |
Successfully pulled image "quay.io/opendatahub/success-200-isvc:odh-pr-1443" in 13.775s (13.775s including waiting). Image size: 1335781480 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b9dc-predictor-7c988b6956-f7lkv |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b9dc-predictor-7c988b6956-f7lkv |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b9dc-predictor-7c988b6956-f7lkv |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.308s (2.308s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b9dc-predictor-68868569ff-tdd6b |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b9dc-predictor-68868569ff-tdd6b |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.269s (2.269s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b9dc-predictor-68868569ff-tdd6b |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7db9dc6879-jr686 |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1443" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-p5mg2 |
Pulling |
Pulling image "kserve/xgbserver:latest" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-56dcdd5cdf-fdd6m |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1443" in 12.683s (12.683s including waiting). Image size: 1561011115 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-56dcdd5cdf-fdd6m |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-56dcdd5cdf-fdd6m |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-56dcdd5cdf-fdd6m |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-56dcdd5cdf-fdd6m |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-56dcdd5cdf-fdd6m |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-56dcdd5cdf-fdd6m |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.195s (2.195s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7db9dc6879-jr686 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7db9dc6879-jr686 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7db9dc6879-jr686 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7db9dc6879-jr686 |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1443" in 5.547s (5.547s including waiting). Image size: 1561011115 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7db9dc6879-jr686 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7db9dc6879-jr686 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-p5mg2 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-p5mg2 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-p5mg2 |
Pulled |
Successfully pulled image "kserve/xgbserver:latest" in 17.491s (17.491s including waiting). Image size: 1306417402 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-p5mg2 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-p5mg2 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-p5mg2 |
Created |
Created container: kube-rbac-proxy | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1b9dc-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1b9dc-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1b9dc-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1b9dc-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b9dc-predictor-68868569ff-tdd6b |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.17:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-1b9dc |
InferenceServiceReady |
InferenceService [error-404-isvc-1b9dc] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-1b9dc |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-1b9dc |
InferenceServiceReady |
InferenceService [success-200-isvc-1b9dc] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-1b9dc |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1b9dc-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1b9dc-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x8) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-56dcdd5cdf-fdd6m |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.28:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
switch-graph-1b9dc-5995d747dd-q5dbh |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-1b9dc-serving-cert" not found | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-1b9dc |
InferenceGraphReady |
InferenceGraph [switch-graph-1b9dc] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-1b9dc |
ScalingReplicaSet |
Scaled up replica set switch-graph-1b9dc-5995d747dd from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-1b9dc-5995d747dd |
SuccessfulCreate |
Created pod: switch-graph-1b9dc-5995d747dd-q5dbh | |
kserve-ci-e2e-test |
multus |
switch-graph-1b9dc-5995d747dd-q5dbh |
AddedInterface |
Add eth0 [10.133.0.29/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-1b9dc-5995d747dd-q5dbh |
Pulling |
Pulling image "quay.io/opendatahub/kserve-router@sha256:f3ef58ae57fd8b984890d9aeb7935958f0ea100fbddeb0833d76b7e2007120ff" | |
kserve-ci-e2e-test |
kubelet |
switch-graph-1b9dc-5995d747dd-q5dbh |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-router@sha256:f3ef58ae57fd8b984890d9aeb7935958f0ea100fbddeb0833d76b7e2007120ff" in 1.807s (1.807s including waiting). Image size: 216346192 bytes. | |
kserve-ci-e2e-test |
kubelet |
switch-graph-1b9dc-5995d747dd-q5dbh |
Started |
Started container switch-graph-1b9dc | |
kserve-ci-e2e-test |
kubelet |
switch-graph-1b9dc-5995d747dd-q5dbh |
Created |
Created container: switch-graph-1b9dc | |
kserve-ci-e2e-test |
kubelet |
switch-graph-1b9dc-5995d747dd-q5dbh |
Killing |
Stopping container switch-graph-1b9dc | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1b9dc-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1b9dc-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-1] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-e938d-predictor-8478f5b96 |
SuccessfulCreate |
Created pod: error-404-isvc-e938d-predictor-8478f5b96-brdgr | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e938d |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-e938d": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e938d |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-e938d": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-e938d": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-e938d-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-e938d-predictor-8478f5b96 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e938d-predictor-858f86b7bc-ckndq |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-e938d-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e938d-predictor-8478f5b96-brdgr |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-e938d-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-e938d-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-e938d-predictor-858f86b7bc from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-e938d-predictor-858f86b7bc |
SuccessfulCreate |
Created pod: success-200-isvc-e938d-predictor-858f86b7bc-ckndq | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b9dc-predictor-68868569ff-tdd6b |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b9dc-predictor-68868569ff-tdd6b |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1b9dc-predictor-68868569ff-tdd6b |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.17:8643/healthz": dial tcp 10.132.0.17:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b9dc-predictor-7c988b6956-f7lkv |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b9dc-predictor-7c988b6956-f7lkv |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e938d-predictor-858f86b7bc-ckndq |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e938d-predictor-8478f5b96-brdgr |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e938d-predictor-8478f5b96-brdgr |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e938d-predictor-8478f5b96-brdgr |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-2] is Ready | |
kserve-ci-e2e-test |
multus |
success-200-isvc-e938d-predictor-858f86b7bc-ckndq |
AddedInterface |
Add eth0 [10.132.0.19/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e938d-predictor-8478f5b96-brdgr |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e938d-predictor-8478f5b96-brdgr |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1443" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e938d-predictor-858f86b7bc-ckndq |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e938d-predictor-858f86b7bc-ckndq |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-e938d-predictor-8478f5b96-brdgr |
AddedInterface |
Add eth0 [10.134.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e938d-predictor-8478f5b96-brdgr |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e938d-predictor-858f86b7bc-ckndq |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1443" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e938d-predictor-858f86b7bc-ckndq |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e938d-predictor-858f86b7bc-ckndq |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b9dc-predictor-7c988b6956-f7lkv |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.29:8643/healthz": dial tcp 10.134.0.29:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-1b9dc-predictor-7c988b6956-f7lkv |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.29:8080: connect: connection refused |
| (x10) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
InferenceServiceReady |
InferenceService [isvc-xgboost-graph] is Ready | |
| (x10) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-1b9dc-5995d747dd-q5dbh |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
model-chainer |
InferenceGraphReady |
InferenceGraph [model-chainer] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
model-chainer |
ScalingReplicaSet |
Scaled up replica set model-chainer-85d456f99f from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
model-chainer-85d456f99f |
SuccessfulCreate |
Created pod: model-chainer-85d456f99f-98r87 | |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
model-chainer-85d456f99f-98r87 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "model-chainer-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
model-chainer-85d456f99f-98r87 |
Created |
Created container: model-chainer | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e938d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
model-chainer-85d456f99f-98r87 |
Started |
Started container model-chainer | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e938d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
multus |
model-chainer-85d456f99f-98r87 |
AddedInterface |
Add eth0 [10.133.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
model-chainer-85d456f99f-98r87 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:f3ef58ae57fd8b984890d9aeb7935958f0ea100fbddeb0833d76b7e2007120ff" already present on machine | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-f8431-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-f8431-predictor-6bcffc9d57 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-p5mg2 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-p5mg2 |
Killing |
Stopping container kube-rbac-proxy | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-e938d-predictor-858f86b7bc-ckndq |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.19:8080: connect: connection refused |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-f8431-predictor-6bcffc9d57 |
SuccessfulCreate |
Created pod: success-200-isvc-f8431-predictor-6bcffc9d57-nq2nz | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7db9dc6879-jr686 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7db9dc6879-jr686 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
model-chainer-85d456f99f-98r87 |
Killing |
Stopping container model-chainer | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f8431-predictor-6bcffc9d57-nq2nz |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1443" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-56dcdd5cdf-fdd6m |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f8431-predictor-6bcffc9d57-nq2nz |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f8431-predictor-6bcffc9d57-nq2nz |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f8431-predictor-6bcffc9d57-nq2nz |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f8431-predictor-6b847f4b94-q6dxt |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-f8431-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f8431-predictor-6bcffc9d57-nq2nz |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
success-200-isvc-f8431-predictor-6bcffc9d57-nq2nz |
AddedInterface |
Add eth0 [10.132.0.20/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f8431-predictor-6bcffc9d57-nq2nz |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-f8431-predictor-6b847f4b94 |
SuccessfulCreate |
Created pod: error-404-isvc-f8431-predictor-6b847f4b94-q6dxt | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-f8431-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-f8431-predictor-6b847f4b94 from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f8431 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-f8431": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-f8431": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f8431 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-f8431": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-56dcdd5cdf-fdd6m |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f8431-predictor-6b847f4b94-q6dxt |
Started |
Started container kserve-container | |
| (x9) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7db9dc6879-jr686 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.30:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7db9dc6879-jr686 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.30:8643/healthz": dial tcp 10.134.0.30:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f8431-predictor-6b847f4b94-q6dxt |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f8431-predictor-6b847f4b94-q6dxt |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f8431-predictor-6b847f4b94-q6dxt |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f8431-predictor-6b847f4b94-q6dxt |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f8431-predictor-6b847f4b94-q6dxt |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1443" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-f8431-predictor-6b847f4b94-q6dxt |
AddedInterface |
Add eth0 [10.134.0.32/23] from ovn-kubernetes | |
| (x9) | kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-p5mg2 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.18:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-669d8d6456-p5mg2 |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.18:8643/healthz": dial tcp 10.132.0.18:8643: connect: connection refused | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e938d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e938d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e938d |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e938d |
InferenceServiceReady |
InferenceService [error-404-isvc-e938d] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e938d |
InferenceServiceReady |
InferenceService [success-200-isvc-e938d] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e938d |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-e938d |
InferenceGraphReady |
InferenceGraph [switch-graph-e938d] is Ready | |
kserve-ci-e2e-test |
kubelet |
switch-graph-e938d-798f566f4f-4lmhz |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-e938d-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-e938d |
ScalingReplicaSet |
Scaled up replica set switch-graph-e938d-798f566f4f from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-e938d-798f566f4f |
SuccessfulCreate |
Created pod: switch-graph-e938d-798f566f4f-4lmhz | |
kserve-ci-e2e-test |
multus |
switch-graph-e938d-798f566f4f-4lmhz |
AddedInterface |
Add eth0 [10.133.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-e938d-798f566f4f-4lmhz |
Created |
Created container: switch-graph-e938d | |
kserve-ci-e2e-test |
kubelet |
switch-graph-e938d-798f566f4f-4lmhz |
Started |
Started container switch-graph-e938d | |
kserve-ci-e2e-test |
kubelet |
switch-graph-e938d-798f566f4f-4lmhz |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:f3ef58ae57fd8b984890d9aeb7935958f0ea100fbddeb0833d76b7e2007120ff" already present on machine | |
| (x6) | kserve-ci-e2e-test |
kubelet |
model-chainer-85d456f99f-98r87 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f8431-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f8431-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f8431-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f8431-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e938d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e938d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e938d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-e938d |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-e938d |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e938d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-f8431-predictor-6bcffc9d57-nq2nz |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.20:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-f8431-predictor-6b847f4b94-q6dxt |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.32:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f8431 |
InferenceServiceReady |
InferenceService [error-404-isvc-f8431] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-f8431 |
InferenceServiceReady |
InferenceService [success-200-isvc-f8431] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-f8431 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f8431 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-e938d |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-e938d |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-f8431-7f98d55687 |
SuccessfulCreate |
Created pod: sequence-graph-f8431-7f98d55687-f4dl5 | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-f8431 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-f8431-7f98d55687 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-f8431 |
InferenceGraphReady |
InferenceGraph [sequence-graph-f8431] is Ready | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-f8431-7f98d55687-f4dl5 |
Started |
Started container sequence-graph-f8431 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f8431-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
sequence-graph-f8431-7f98d55687-f4dl5 |
Created |
Created container: sequence-graph-f8431 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-f8431-7f98d55687-f4dl5 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:f3ef58ae57fd8b984890d9aeb7935958f0ea100fbddeb0833d76b7e2007120ff" already present on machine | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f8431-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f8431-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
multus |
sequence-graph-f8431-7f98d55687-f4dl5 |
AddedInterface |
Add eth0 [10.133.0.32/23] from ovn-kubernetes | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f8431-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-f8431 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-f8431 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
switch-graph-e938d-798f566f4f-4lmhz |
Killing |
Stopping container switch-graph-e938d | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e938d-predictor-858f86b7bc-ckndq |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e938d-predictor-8478f5b96-brdgr |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-5c921-predictor-8548d846d5 |
SuccessfulCreate |
Created pod: success-200-isvc-5c921-predictor-8548d846d5-287kf | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-5c921 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-5c921": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-5c921": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-5c921 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-5c921": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e938d-predictor-8478f5b96-brdgr |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e938d-predictor-858f86b7bc-ckndq |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-5c921-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-5c921-predictor-8548d846d5 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5c921-predictor-78cfcc75d6-6fxpn |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1443" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-5c921-predictor-78cfcc75d6-6fxpn |
AddedInterface |
Add eth0 [10.134.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-5c921-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-5c921-predictor-78cfcc75d6 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5c921-predictor-78cfcc75d6-6fxpn |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5c921-predictor-78cfcc75d6-6fxpn |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5c921-predictor-78cfcc75d6-6fxpn |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5c921-predictor-78cfcc75d6-6fxpn |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5c921-predictor-78cfcc75d6-6fxpn |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-5c921 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-5c921": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-5c921": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-5c921 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-5c921": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5c921-predictor-8548d846d5-287kf |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-5c921-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-5c921-predictor-78cfcc75d6 |
SuccessfulCreate |
Created pod: error-404-isvc-5c921-predictor-78cfcc75d6-6fxpn | |
kserve-ci-e2e-test |
multus |
success-200-isvc-5c921-predictor-8548d846d5-287kf |
AddedInterface |
Add eth0 [10.132.0.21/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5c921-predictor-8548d846d5-287kf |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5c921-predictor-8548d846d5-287kf |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5c921-predictor-8548d846d5-287kf |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5c921-predictor-8548d846d5-287kf |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1443" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5c921-predictor-8548d846d5-287kf |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5c921-predictor-8548d846d5-287kf |
Started |
Started container kserve-container | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-e938d-predictor-8478f5b96-brdgr |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.31:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e938d-predictor-8478f5b96-brdgr |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.31:8643/healthz": dial tcp 10.134.0.31:8643: connect: connection refused | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-5c921-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-5c921-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-e938d-798f566f4f-4lmhz |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-5c921-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-5c921-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-5c921-predictor-8548d846d5-287kf |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.21:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-5c921-predictor-78cfcc75d6-6fxpn |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.33:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
sequence-graph-f8431-7f98d55687-f4dl5 |
Killing |
Stopping container sequence-graph-f8431 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f8431-predictor-6bcffc9d57-nq2nz |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-32c7c |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-32c7c": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-32c7c": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f8431-predictor-6bcffc9d57-nq2nz |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-32c7c-predictor-57df59874f |
SuccessfulCreate |
Created pod: success-200-isvc-32c7c-predictor-57df59874f-7vtc8 | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-32c7c-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-32c7c-predictor-57df59874f from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-32c7c |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-32c7c": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-32c7c |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-32c7c": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f8431-predictor-6b847f4b94-q6dxt |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-32c7c-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-32c7c-predictor-69fdd96fd6 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-32c7c-predictor-57df59874f-7vtc8 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1443" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-32c7c-predictor-57df59874f-7vtc8 |
AddedInterface |
Add eth0 [10.132.0.22/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-32c7c-predictor-57df59874f-7vtc8 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-32c7c-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-32c7c |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-32c7c": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-32c7c": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-32c7c-predictor-69fdd96fd6 |
SuccessfulCreate |
Created pod: error-404-isvc-32c7c-predictor-69fdd96fd6-v6c7z | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f8431-predictor-6b847f4b94-q6dxt |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-32c7c-predictor-69fdd96fd6-v6c7z |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-32c7c-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
multus |
error-404-isvc-32c7c-predictor-69fdd96fd6-v6c7z |
AddedInterface |
Add eth0 [10.134.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-32c7c-predictor-57df59874f-7vtc8 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-32c7c-predictor-69fdd96fd6-v6c7z |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-32c7c-predictor-69fdd96fd6-v6c7z |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-32c7c-predictor-69fdd96fd6-v6c7z |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-32c7c-predictor-69fdd96fd6-v6c7z |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-32c7c-predictor-69fdd96fd6-v6c7z |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-32c7c-predictor-57df59874f-7vtc8 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-32c7c-predictor-57df59874f-7vtc8 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-32c7c-predictor-69fdd96fd6-v6c7z |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1443" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-32c7c-predictor-57df59874f-7vtc8 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-32c7c-predictor-57df59874f-7vtc8 |
Created |
Created container: kube-rbac-proxy | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-5c921 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-5c921 |
InferenceServiceReady |
InferenceService [success-200-isvc-5c921] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-5c921 |
InferenceServiceReady |
InferenceService [error-404-isvc-5c921] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-5c921 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-5c921-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-5c921-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-5c921-c79bf4c85-pqgsq |
Created |
Created container: ensemble-graph-5c921 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-5c921-c79bf4c85-pqgsq |
Started |
Started container ensemble-graph-5c921 | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-5c921-c79bf4c85 |
SuccessfulCreate |
Created pod: ensemble-graph-5c921-c79bf4c85-pqgsq | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-5c921-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-5c921 |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-5c921-c79bf4c85 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-5c921-c79bf4c85-pqgsq |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:f3ef58ae57fd8b984890d9aeb7935958f0ea100fbddeb0833d76b7e2007120ff" already present on machine | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-5c921 |
InferenceGraphReady |
InferenceGraph [ensemble-graph-5c921] is Ready | |
kserve-ci-e2e-test |
multus |
ensemble-graph-5c921-c79bf4c85-pqgsq |
AddedInterface |
Add eth0 [10.133.0.33/23] from ovn-kubernetes | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-5c921-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-f8431-7f98d55687-f4dl5 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4aba3-predictor-6dcd794499-dzrk9 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-4aba3 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-4aba3": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-4aba3 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-4aba3": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5c921-predictor-8548d846d5-287kf |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5c921-predictor-8548d846d5-287kf |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-4aba3-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-4aba3-predictor-6f567c46fb from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-4aba3-predictor-6f567c46fb |
SuccessfulCreate |
Created pod: error-404-isvc-4aba3-predictor-6f567c46fb-kdltw | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4aba3-predictor-6f567c46fb-kdltw |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-4aba3-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5c921-predictor-78cfcc75d6-6fxpn |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5c921-predictor-78cfcc75d6-6fxpn |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-4aba3 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-4aba3": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-4aba3": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-4aba3 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-4aba3": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-4aba3": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-4aba3-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-4aba3-predictor-6dcd794499 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-4aba3-predictor-6dcd794499 |
SuccessfulCreate |
Created pod: success-200-isvc-4aba3-predictor-6dcd794499-dzrk9 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4aba3-predictor-6dcd794499-dzrk9 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-4aba3-predictor-6dcd794499-dzrk9 |
AddedInterface |
Add eth0 [10.132.0.23/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4aba3-predictor-6dcd794499-dzrk9 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1443" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4aba3-predictor-6dcd794499-dzrk9 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4aba3-predictor-6dcd794499-dzrk9 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-5c921-c79bf4c85-pqgsq |
Killing |
Stopping container ensemble-graph-5c921 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4aba3-predictor-6dcd794499-dzrk9 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4aba3-predictor-6f567c46fb-kdltw |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4aba3-predictor-6f567c46fb-kdltw |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1443" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-4aba3-predictor-6f567c46fb-kdltw |
AddedInterface |
Add eth0 [10.134.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4aba3-predictor-6f567c46fb-kdltw |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4aba3-predictor-6f567c46fb-kdltw |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4aba3-predictor-6f567c46fb-kdltw |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4aba3-predictor-6f567c46fb-kdltw |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5c921-predictor-8548d846d5-287kf |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.21:8643/healthz": dial tcp 10.132.0.21:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5c921-predictor-78cfcc75d6-6fxpn |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.33:8643/healthz": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-32c7c-predictor-57df59874f-7vtc8 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.22:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-32c7c-predictor-69fdd96fd6-v6c7z |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.34:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-32c7c-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-32c7c-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-32c7c-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-32c7c-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-32c7c |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-32c7c |
InferenceServiceReady |
InferenceService [error-404-isvc-32c7c] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-32c7c |
InferenceServiceReady |
InferenceService [success-200-isvc-32c7c] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-32c7c |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-5c921-c79bf4c85-pqgsq |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-32c7c-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-32c7c-6ff7f8b44c |
SuccessfulCreate |
Created pod: sequence-graph-32c7c-6ff7f8b44c-rwdm4 | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-32c7c |
ScalingReplicaSet |
Scaled up replica set sequence-graph-32c7c-6ff7f8b44c from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-32c7c |
InferenceGraphReady |
InferenceGraph [sequence-graph-32c7c] is Ready | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-32c7c-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4aba3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-32c7c-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4aba3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
kubelet |
sequence-graph-32c7c-6ff7f8b44c-rwdm4 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "sequence-graph-32c7c-serving-cert" not found |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4aba3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-32c7c-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4aba3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
sequence-graph-32c7c-6ff7f8b44c-rwdm4 |
Started |
Started container sequence-graph-32c7c | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-32c7c-6ff7f8b44c-rwdm4 |
Created |
Created container: sequence-graph-32c7c | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-32c7c-6ff7f8b44c-rwdm4 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:f3ef58ae57fd8b984890d9aeb7935958f0ea100fbddeb0833d76b7e2007120ff" already present on machine | |
kserve-ci-e2e-test |
multus |
sequence-graph-32c7c-6ff7f8b44c-rwdm4 |
AddedInterface |
Add eth0 [10.133.0.34/23] from ovn-kubernetes | |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-4aba3-predictor-6f567c46fb-kdltw |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.35:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e6fe-predictor-89f79697d-42755 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e6fe-predictor-89f79697d-42755 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-32c7c-predictor-57df59874f-7vtc8 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-32c7c-predictor-69fdd96fd6-v6c7z |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-32c7c-predictor-69fdd96fd6-v6c7z |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-2e6fe |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-2e6fe": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-32c7c-predictor-57df59874f-7vtc8 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-2e6fe |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-2e6fe": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-2e6fe": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-2e6fe-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-2e6fe-predictor-89f79697d from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-32c7c-6ff7f8b44c-rwdm4 |
Killing |
Stopping container sequence-graph-32c7c | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-2e6fe-predictor-89f79697d |
SuccessfulCreate |
Created pod: error-404-isvc-2e6fe-predictor-89f79697d-42755 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e6fe-predictor-89f79697d-42755 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-2e6fe-predictor-66c4d45c7c-xmb5t |
AddedInterface |
Add eth0 [10.132.0.24/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e6fe-predictor-66c4d45c7c-xmb5t |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1443" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e6fe-predictor-66c4d45c7c-xmb5t |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e6fe-predictor-66c4d45c7c-xmb5t |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e6fe-predictor-66c4d45c7c-xmb5t |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e6fe-predictor-66c4d45c7c-xmb5t |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e6fe-predictor-66c4d45c7c-xmb5t |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-2e6fe |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-2e6fe": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e6fe-predictor-89f79697d-42755 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e6fe-predictor-89f79697d-42755 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e6fe-predictor-89f79697d-42755 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1443" already present on machine | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-2e6fe-predictor-66c4d45c7c |
SuccessfulCreate |
Created pod: success-200-isvc-2e6fe-predictor-66c4d45c7c-xmb5t | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-2e6fe-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-2e6fe-predictor-66c4d45c7c from 0 to 1 | |
kserve-ci-e2e-test |
multus |
error-404-isvc-2e6fe-predictor-89f79697d-42755 |
AddedInterface |
Add eth0 [10.134.0.36/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-2e6fe |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-2e6fe": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-2e6fe": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-4aba3 |
InferenceServiceReady |
InferenceService [error-404-isvc-4aba3] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-4aba3 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-4aba3 |
InferenceServiceReady |
InferenceService [success-200-isvc-4aba3] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-4aba3 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-4aba3-5f6b4f4ff7-vzzq7 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "ensemble-graph-4aba3-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-4aba3-5f6b4f4ff7 |
SuccessfulCreate |
Created pod: ensemble-graph-4aba3-5f6b4f4ff7-vzzq7 | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-4aba3 |
InferenceGraphReady |
InferenceGraph [ensemble-graph-4aba3] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-4aba3 |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-4aba3-5f6b4f4ff7 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-4aba3-5f6b4f4ff7-vzzq7 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:f3ef58ae57fd8b984890d9aeb7935958f0ea100fbddeb0833d76b7e2007120ff" already present on machine | |
kserve-ci-e2e-test |
multus |
ensemble-graph-4aba3-5f6b4f4ff7-vzzq7 |
AddedInterface |
Add eth0 [10.133.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-4aba3-5f6b4f4ff7-vzzq7 |
Created |
Created container: ensemble-graph-4aba3 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-4aba3-5f6b4f4ff7-vzzq7 |
Started |
Started container ensemble-graph-4aba3 | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-32c7c-6ff7f8b44c-rwdm4 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4aba3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4aba3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4aba3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4aba3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2e6fe-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2e6fe-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2e6fe-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2e6fe-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-2e6fe |
InferenceServiceReady |
InferenceService [error-404-isvc-2e6fe] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-2e6fe |
InferenceServiceReady |
InferenceService [success-200-isvc-2e6fe] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-2e6fe |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-2e6fe |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-4aba3 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-4aba3 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-2e6fe-dcb8bc6f8 |
SuccessfulCreate |
Created pod: sequence-graph-2e6fe-dcb8bc6f8-mvk48 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-2e6fe-dcb8bc6f8-mvk48 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "sequence-graph-2e6fe-serving-cert" not found | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-2e6fe |
InferenceGraphReady |
InferenceGraph [sequence-graph-2e6fe] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-2e6fe |
ScalingReplicaSet |
Scaled up replica set sequence-graph-2e6fe-dcb8bc6f8 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-2e6fe-dcb8bc6f8-mvk48 |
Started |
Started container sequence-graph-2e6fe | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-2e6fe-dcb8bc6f8-mvk48 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:f3ef58ae57fd8b984890d9aeb7935958f0ea100fbddeb0833d76b7e2007120ff" already present on machine | |
kserve-ci-e2e-test |
multus |
sequence-graph-2e6fe-dcb8bc6f8-mvk48 |
AddedInterface |
Add eth0 [10.133.0.36/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-2e6fe-dcb8bc6f8-mvk48 |
Created |
Created container: sequence-graph-2e6fe | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2e6fe-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2e6fe-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2e6fe-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2e6fe-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-2e6fe |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-2e6fe |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-44b1e-predictor-648b8cfbf8-mdq6n |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1443" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4aba3-predictor-6f567c46fb-kdltw |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-44b1e |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-44b1e": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-44b1e |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-44b1e": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-44b1e": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-44b1e-predictor-589485df45-9jvsp |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-44b1e-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-44b1e-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-44b1e-predictor-589485df45 from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-44b1e-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-44b1e-predictor-648b8cfbf8 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-44b1e-predictor-589485df45 |
SuccessfulCreate |
Created pod: success-200-isvc-44b1e-predictor-589485df45-9jvsp | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4aba3-predictor-6f567c46fb-kdltw |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-44b1e-predictor-648b8cfbf8-mdq6n |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-4aba3-5f6b4f4ff7-vzzq7 |
Killing |
Stopping container ensemble-graph-4aba3 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-44b1e-predictor-648b8cfbf8 |
SuccessfulCreate |
Created pod: error-404-isvc-44b1e-predictor-648b8cfbf8-mdq6n | |
kserve-ci-e2e-test |
multus |
error-404-isvc-44b1e-predictor-648b8cfbf8-mdq6n |
AddedInterface |
Add eth0 [10.134.0.37/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-44b1e-predictor-648b8cfbf8-mdq6n |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-44b1e-predictor-648b8cfbf8-mdq6n |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4aba3-predictor-6dcd794499-dzrk9 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4aba3-predictor-6dcd794499-dzrk9 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-44b1e-predictor-648b8cfbf8-mdq6n |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-44b1e-predictor-648b8cfbf8-mdq6n |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-44b1e-predictor-589485df45-9jvsp |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-44b1e-predictor-589485df45-9jvsp |
AddedInterface |
Add eth0 [10.132.0.25/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-44b1e-predictor-589485df45-9jvsp |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1443" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-44b1e-predictor-589485df45-9jvsp |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-44b1e-predictor-589485df45-9jvsp |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-44b1e-predictor-589485df45-9jvsp |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-44b1e-predictor-589485df45-9jvsp |
Created |
Created container: kserve-container | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-4aba3-predictor-6dcd794499-dzrk9 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.23:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4aba3-predictor-6dcd794499-dzrk9 |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.23:8643/healthz": dial tcp 10.132.0.23:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-4aba3-5f6b4f4ff7-vzzq7 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-44b1e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-44b1e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-44b1e-predictor-589485df45-9jvsp |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.25:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-44b1e-predictor-648b8cfbf8-mdq6n |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.37:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
sequence-graph-2e6fe-dcb8bc6f8-mvk48 |
Killing |
Stopping container sequence-graph-2e6fe | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-d0532-predictor-7bbf598d6b |
SuccessfulCreate |
Created pod: error-404-isvc-d0532-predictor-7bbf598d6b-xswpz | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d0532-predictor-6cfcdcc985-fjlp5 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d0532-predictor-7bbf598d6b-xswpz |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d0532-predictor-6cfcdcc985-fjlp5 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d0532-predictor-7bbf598d6b-xswpz |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d0532-predictor-7bbf598d6b-xswpz |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d0532-predictor-7bbf598d6b-xswpz |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-d0532-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-d0532-predictor-7bbf598d6b from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d0532-predictor-7bbf598d6b-xswpz |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d0532-predictor-6cfcdcc985-fjlp5 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d0532-predictor-7bbf598d6b-xswpz |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1443" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-d0532-predictor-7bbf598d6b-xswpz |
AddedInterface |
Add eth0 [10.134.0.38/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e6fe-predictor-89f79697d-42755 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d0532 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-d0532": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e6fe-predictor-89f79697d-42755 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d0532 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-d0532": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-d0532": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d0532-predictor-6cfcdcc985-fjlp5 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d0532-predictor-6cfcdcc985-fjlp5 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d0532 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-d0532": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d0532 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-d0532": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-d0532": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-d0532-predictor-6cfcdcc985 |
SuccessfulCreate |
Created pod: success-200-isvc-d0532-predictor-6cfcdcc985-fjlp5 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d0532-predictor-6cfcdcc985-fjlp5 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1443" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-d0532-predictor-6cfcdcc985-fjlp5 |
AddedInterface |
Add eth0 [10.132.0.26/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-d0532-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-d0532-predictor-6cfcdcc985 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e6fe-predictor-66c4d45c7c-xmb5t |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e6fe-predictor-66c4d45c7c-xmb5t |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e6fe-predictor-89f79697d-42755 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.36:8643/healthz": dial tcp 10.134.0.36:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e6fe-predictor-89f79697d-42755 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.36:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e6fe-predictor-66c4d45c7c-xmb5t |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.24:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e6fe-predictor-66c4d45c7c-xmb5t |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.24:8643/healthz": dial tcp 10.132.0.24:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-44b1e |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-44b1e |
InferenceServiceReady |
InferenceService [success-200-isvc-44b1e] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-44b1e |
InferenceServiceReady |
InferenceService [error-404-isvc-44b1e] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-44b1e |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
kubelet |
splitter-graph-44b1e-98dcdcd84-rr45w |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "splitter-graph-44b1e-serving-cert" not found | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-44b1e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-44b1e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-44b1e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-44b1e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-44b1e |
InferenceGraphReady |
InferenceGraph [splitter-graph-44b1e] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-44b1e |
ScalingReplicaSet |
Scaled up replica set splitter-graph-44b1e-98dcdcd84 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-44b1e-98dcdcd84 |
SuccessfulCreate |
Created pod: splitter-graph-44b1e-98dcdcd84-rr45w | |
kserve-ci-e2e-test |
multus |
splitter-graph-44b1e-98dcdcd84-rr45w |
AddedInterface |
Add eth0 [10.133.0.37/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-44b1e-98dcdcd84-rr45w |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:f3ef58ae57fd8b984890d9aeb7935958f0ea100fbddeb0833d76b7e2007120ff" already present on machine | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-44b1e-98dcdcd84-rr45w |
Created |
Created container: splitter-graph-44b1e | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-44b1e-98dcdcd84-rr45w |
Started |
Started container splitter-graph-44b1e | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-2e6fe-dcb8bc6f8-mvk48 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-be7a5 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-be7a5": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-be7a5": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-44b1e-98dcdcd84-rr45w |
Killing |
Stopping container splitter-graph-44b1e | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d0532-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d0532-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-be7a5-predictor-5c7766ddbb-8xswq |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-be7a5-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-be7a5-predictor-6654cff85-qtt26 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-be7a5-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-44b1e-predictor-589485df45-9jvsp |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-44b1e-predictor-589485df45-9jvsp |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-be7a5-predictor-5c7766ddbb |
SuccessfulCreate |
Created pod: error-404-isvc-be7a5-predictor-5c7766ddbb-8xswq | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-44b1e-predictor-648b8cfbf8-mdq6n |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-be7a5 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-be7a5": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-be7a5 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-be7a5": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-44b1e-predictor-648b8cfbf8-mdq6n |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-be7a5-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-be7a5-predictor-5c7766ddbb from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-be7a5-predictor-6654cff85 |
SuccessfulCreate |
Created pod: success-200-isvc-be7a5-predictor-6654cff85-qtt26 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-be7a5 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-be7a5": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-be7a5": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-be7a5-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-be7a5-predictor-6654cff85 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-be7a5-predictor-6654cff85-qtt26 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1443" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-be7a5-predictor-5c7766ddbb-8xswq |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1443" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-be7a5-predictor-5c7766ddbb-8xswq |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-be7a5-predictor-5c7766ddbb-8xswq |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-be7a5-predictor-5c7766ddbb-8xswq |
AddedInterface |
Add eth0 [10.134.0.39/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-be7a5-predictor-6654cff85-qtt26 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-be7a5-predictor-5c7766ddbb-8xswq |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-be7a5-predictor-6654cff85-qtt26 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-be7a5-predictor-5c7766ddbb-8xswq |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-be7a5-predictor-5c7766ddbb-8xswq |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-be7a5-predictor-6654cff85-qtt26 |
AddedInterface |
Add eth0 [10.132.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-be7a5-predictor-6654cff85-qtt26 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-be7a5-predictor-6654cff85-qtt26 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-be7a5-predictor-6654cff85-qtt26 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-44b1e-predictor-589485df45-9jvsp |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.25:8643/healthz": dial tcp 10.132.0.25:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-44b1e-predictor-648b8cfbf8-mdq6n |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.37:8643/healthz": dial tcp 10.134.0.37:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-d0532-predictor-7bbf598d6b-xswpz |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.38:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-d0532-predictor-6cfcdcc985-fjlp5 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.26:8080: connect: connection refused |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-be7a5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d0532-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-be7a5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d0532-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d0532 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d0532 |
InferenceServiceReady |
InferenceService [error-404-isvc-d0532] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d0532 |
InferenceServiceReady |
InferenceService [success-200-isvc-d0532] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d0532 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-44b1e-98dcdcd84-rr45w |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-d0532-75644c6c8f |
SuccessfulCreate |
Created pod: switch-graph-d0532-75644c6c8f-ccgwj | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-d0532 |
ScalingReplicaSet |
Scaled up replica set switch-graph-d0532-75644c6c8f from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-d0532 |
InferenceGraphReady |
InferenceGraph [switch-graph-d0532] is Ready | |
kserve-ci-e2e-test |
kubelet |
switch-graph-d0532-75644c6c8f-ccgwj |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-d0532-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
switch-graph-d0532-75644c6c8f-ccgwj |
Created |
Created container: switch-graph-d0532 | |
kserve-ci-e2e-test |
multus |
switch-graph-d0532-75644c6c8f-ccgwj |
AddedInterface |
Add eth0 [10.133.0.38/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-d0532-75644c6c8f-ccgwj |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:f3ef58ae57fd8b984890d9aeb7935958f0ea100fbddeb0833d76b7e2007120ff" already present on machine | |
kserve-ci-e2e-test |
kubelet |
switch-graph-d0532-75644c6c8f-ccgwj |
Started |
Started container switch-graph-d0532 | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-be7a5-predictor-6654cff85-qtt26 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.27:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-be7a5-predictor-5c7766ddbb-8xswq |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.39:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-be7a5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-be7a5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-be7a5 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-be7a5 |
InferenceServiceReady |
InferenceService [success-200-isvc-be7a5] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-be7a5 |
InferenceServiceReady |
InferenceService [error-404-isvc-be7a5] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-be7a5 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d0532-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-be7a5-64dbc87fdf |
SuccessfulCreate |
Created pod: splitter-graph-be7a5-64dbc87fdf-ddr2h | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-be7a5 |
InferenceGraphReady |
InferenceGraph [splitter-graph-be7a5] is Ready | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d0532-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d0532-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d0532-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-be7a5 |
ScalingReplicaSet |
Scaled up replica set splitter-graph-be7a5-64dbc87fdf from 0 to 1 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-d0532 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-d0532 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
multus |
splitter-graph-be7a5-64dbc87fdf-ddr2h |
AddedInterface |
Add eth0 [10.133.0.39/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-be7a5-64dbc87fdf-ddr2h |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:f3ef58ae57fd8b984890d9aeb7935958f0ea100fbddeb0833d76b7e2007120ff" already present on machine | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-be7a5-64dbc87fdf-ddr2h |
Created |
Created container: splitter-graph-be7a5 | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-be7a5-64dbc87fdf-ddr2h |
Started |
Started container splitter-graph-be7a5 | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-be7a5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-be7a5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-be7a5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-be7a5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-be7a5 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-be7a5 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-be7a5-predictor-6654cff85-qtt26 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-be7a5-predictor-6654cff85-qtt26 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-be7a5-64dbc87fdf-ddr2h |
Killing |
Stopping container splitter-graph-be7a5 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-be7a5-predictor-5c7766ddbb-8xswq |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-be7a5-predictor-5c7766ddbb-8xswq |
Killing |
Stopping container kserve-container | |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-be7a5-64dbc87fdf-ddr2h |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
switch-graph-d0532-75644c6c8f-ccgwj |
Killing |
Stopping container switch-graph-d0532 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d0532-predictor-6cfcdcc985-fjlp5 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d0532-predictor-7bbf598d6b-xswpz |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d0532-predictor-7bbf598d6b-xswpz |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d0532-predictor-6cfcdcc985-fjlp5 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d0532-predictor-7bbf598d6b-xswpz |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.38:8643/healthz": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) | |
| (x5) | kserve-ci-e2e-test |
kubelet |
switch-graph-d0532-75644c6c8f-ccgwj |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |