| Time | Namespace | Component | RelatedObject | Reason | Message |
|---|---|---|---|---|---|
kserve-ci-e2e-test |
splitter-graph-f7711-5d9485b744-t55xb |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-f7711-5d9485b744-t55xb to ip-10-0-140-175.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-222a3-predictor-bb95bb5f7-jbnk4 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-222a3-predictor-bb95bb5f7-jbnk4 to ip-10-0-131-51.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-4e070-57b477589f-dnp8r |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-4e070-57b477589f-dnp8r to ip-10-0-140-175.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-e92a5-predictor-5cdccb45b8-bnrjn |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-e92a5-predictor-5cdccb45b8-bnrjn to ip-10-0-131-51.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-8d409-predictor-5fcc67fb8c-l8z5j |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-8d409-predictor-5fcc67fb8c-l8z5j to ip-10-0-131-51.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-4e070-predictor-f797d89df-t5768 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-4e070-predictor-f797d89df-t5768 to ip-10-0-131-51.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-e803a-predictor-658895f866-4669k |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-e803a-predictor-658895f866-4669k to ip-10-0-131-51.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-f7711-predictor-7b6f54fcd8-dkzjv |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-f7711-predictor-7b6f54fcd8-dkzjv to ip-10-0-131-51.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-717c3-b69fd89f-9mwps |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-717c3-b69fd89f-9mwps to ip-10-0-140-175.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-717c3-predictor-7db58fb9f4-htnjs |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-717c3-predictor-7db58fb9f4-htnjs to ip-10-0-131-51.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-af89d-predictor-6bff89d85-j66sp |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-af89d-predictor-6bff89d85-j66sp to ip-10-0-131-51.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-4e070-predictor-6985d68b68-kls5q |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-4e070-predictor-6985d68b68-kls5q to ip-10-0-131-51.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-e92a5-6d7f7969cc-8q5zh |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-e92a5-6d7f7969cc-8q5zh to ip-10-0-140-175.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-af89d-db7874558-kldsc |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-af89d-db7874558-kldsc to ip-10-0-140-175.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-a647d-7bc89888f9-ftrq7 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-a647d-7bc89888f9-ftrq7 to ip-10-0-140-175.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-0b9d4-7db5cc7799-hgsps |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-0b9d4-7db5cc7799-hgsps to ip-10-0-140-175.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-222a3-predictor-6d5d87c54d-2tg26 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-222a3-predictor-6d5d87c54d-2tg26 to ip-10-0-131-51.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-f7711-predictor-6d447c4c9c-vwjvk |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-f7711-predictor-6d447c4c9c-vwjvk to ip-10-0-131-51.ec2.internal | ||
kserve-ci-e2e-test |
isvc-xgboost-graph-predictor-555fc69d56-srpkx |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-srpkx to ip-10-0-130-25.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-e92a5-predictor-599bf66b77-hrnbd |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-e92a5-predictor-599bf66b77-hrnbd to ip-10-0-131-51.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-222a3-5f5974b4b4-bb4dl |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-222a3-5f5974b4b4-bb4dl to ip-10-0-140-175.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-8d409-85cbb7d94-dp7l8 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-8d409-85cbb7d94-dp7l8 to ip-10-0-140-175.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-e803a-55fbf78854-lnmxf |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-e803a-55fbf78854-lnmxf to ip-10-0-140-175.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-0b9d4-predictor-7fdbfdf779-f5hq5 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-0b9d4-predictor-7fdbfdf779-f5hq5 to ip-10-0-131-51.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-af89d-predictor-6fc7bd7dcb-2989w |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-af89d-predictor-6fc7bd7dcb-2989w to ip-10-0-131-51.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-a647d-predictor-5474579676-s27ps |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-a647d-predictor-5474579676-s27ps to ip-10-0-131-51.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-8d409-predictor-5f6cb868f4-8njwq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-8d409-predictor-5f6cb868f4-8njwq to ip-10-0-131-51.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-e803a-predictor-76bd58685f-h5jzr |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-e803a-predictor-76bd58685f-h5jzr to ip-10-0-131-51.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-717c3-predictor-67f5fb9455-8qnmw |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-717c3-predictor-67f5fb9455-8qnmw to ip-10-0-131-51.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-a647d-predictor-784bd7c46f-s7brw |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-a647d-predictor-784bd7c46f-s7brw to ip-10-0-131-51.ec2.internal | ||
kserve-ci-e2e-test |
model-chainer-5595dc65c9-w4vhv |
Scheduled |
Successfully assigned kserve-ci-e2e-test/model-chainer-5595dc65c9-w4vhv to ip-10-0-140-175.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-2-predictor-5bcddb87c8-46rgb |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bcddb87c8-46rgb to ip-10-0-140-175.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-1-predictor-85cb8c5574-2ccq6 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-85cb8c5574-2ccq6 to ip-10-0-130-25.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-0b9d4-predictor-54b67cb57d-48n99 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-0b9d4-predictor-54b67cb57d-48n99 to ip-10-0-131-51.ec2.internal | ||
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-1-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-1-predictor-85cb8c5574 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-717c3-predictor-67f5fb9455 |
SuccessfulCreate |
Created pod: error-404-isvc-717c3-predictor-67f5fb9455-8qnmw | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-717c3-predictor-7db58fb9f4 |
SuccessfulCreate |
Created pod: success-200-isvc-717c3-predictor-7db58fb9f4-htnjs | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-717c3-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-717c3-predictor-67f5fb9455 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-1-predictor-85cb8c5574 |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-1-predictor-85cb8c5574-2ccq6 | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-1-predictor-85cb8c5574-2ccq6 |
AddedInterface |
Add eth0 [10.133.0.22/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-85cb8c5574-2ccq6 |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:527f9c4ef06f4e1d255c6c4b87333be01c67680fe0b3417b8fb7aeb5d5b61bbf" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-85cb8c5574-2ccq6 |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-85cb8c5574-2ccq6 |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-717c3-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-717c3-predictor-7db58fb9f4 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-srpkx |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "isvc-xgboost-graph-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-xgboost-graph-predictor-555fc69d56 |
SuccessfulCreate |
Created pod: isvc-xgboost-graph-predictor-555fc69d56-srpkx | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-2-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-2-predictor-5bcddb87c8 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-717c3-predictor-7db58fb9f4-htnjs |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-717c3-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-2-predictor-5bcddb87c8 |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-2-predictor-5bcddb87c8-46rgb | |
kserve-ci-e2e-test |
deployment-controller |
isvc-xgboost-graph-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-xgboost-graph-predictor-555fc69d56 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-srpkx |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
multus |
isvc-xgboost-graph-predictor-555fc69d56-srpkx |
AddedInterface |
Add eth0 [10.133.0.23/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
success-200-isvc-717c3-predictor-7db58fb9f4-htnjs |
AddedInterface |
Add eth0 [10.134.0.20/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-717c3-predictor-7db58fb9f4-htnjs |
Pulling |
Pulling image "quay.io/opendatahub/success-200-isvc:odh-pr-1813" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-srpkx |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-717c3-predictor-67f5fb9455-8qnmw |
Pulling |
Pulling image "quay.io/opendatahub/error-404-isvc:odh-pr-1813" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-srpkx |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:527f9c4ef06f4e1d255c6c4b87333be01c67680fe0b3417b8fb7aeb5d5b61bbf" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5bcddb87c8-46rgb |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "isvc-sklearn-graph-2-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
multus |
error-404-isvc-717c3-predictor-67f5fb9455-8qnmw |
AddedInterface |
Add eth0 [10.134.0.21/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5bcddb87c8-46rgb |
Pulling |
Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:527f9c4ef06f4e1d255c6c4b87333be01c67680fe0b3417b8fb7aeb5d5b61bbf" | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-2-predictor-5bcddb87c8-46rgb |
AddedInterface |
Add eth0 [10.132.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5bcddb87c8-46rgb |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-85cb8c5574-2ccq6 |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1813" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5bcddb87c8-46rgb |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:527f9c4ef06f4e1d255c6c4b87333be01c67680fe0b3417b8fb7aeb5d5b61bbf" in 3.405s (3.405s including waiting). Image size: 302621120 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5bcddb87c8-46rgb |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-srpkx |
Pulling |
Pulling image "kserve/xgbserver:latest" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5bcddb87c8-46rgb |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1813" | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-717c3-predictor-7db58fb9f4-htnjs |
Pulled |
Successfully pulled image "quay.io/opendatahub/success-200-isvc:odh-pr-1813" in 15.37s (15.37s including waiting). Image size: 1334989926 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-717c3-predictor-7db58fb9f4-htnjs |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-717c3-predictor-7db58fb9f4-htnjs |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-717c3-predictor-67f5fb9455-8qnmw |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-717c3-predictor-67f5fb9455-8qnmw |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-717c3-predictor-7db58fb9f4-htnjs |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-717c3-predictor-67f5fb9455-8qnmw |
Pulled |
Successfully pulled image "quay.io/opendatahub/error-404-isvc:odh-pr-1813" in 15.563s (15.563s including waiting). Image size: 1333940832 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-717c3-predictor-67f5fb9455-8qnmw |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-717c3-predictor-67f5fb9455-8qnmw |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-717c3-predictor-67f5fb9455-8qnmw |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.446s (2.446s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-717c3-predictor-7db58fb9f4-htnjs |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.439s (2.439s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-717c3-predictor-7db58fb9f4-htnjs |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-717c3-predictor-7db58fb9f4-htnjs |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-717c3-predictor-67f5fb9455-8qnmw |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5bcddb87c8-46rgb |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5bcddb87c8-46rgb |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1813" in 12.756s (12.756s including waiting). Image size: 1555733417 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5bcddb87c8-46rgb |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5bcddb87c8-46rgb |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5bcddb87c8-46rgb |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.188s (2.188s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5bcddb87c8-46rgb |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5bcddb87c8-46rgb |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-85cb8c5574-2ccq6 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-85cb8c5574-2ccq6 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-85cb8c5574-2ccq6 |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-85cb8c5574-2ccq6 |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1813" in 23.568s (23.568s including waiting). Image size: 1555733417 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-srpkx |
Pulled |
Successfully pulled image "kserve/xgbserver:latest" in 22.962s (22.962s including waiting). Image size: 1307355896 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-srpkx |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-srpkx |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-srpkx |
Started |
Started container kserve-container | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-srpkx |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-85cb8c5574-2ccq6 |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 3.158s (3.158s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-85cb8c5574-2ccq6 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-85cb8c5574-2ccq6 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-srpkx |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-srpkx |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.776s (2.776s including waiting). Image size: 211946088 bytes. | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-717c3-predictor-7db58fb9f4-htnjs |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.20:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-717c3-predictor-67f5fb9455-8qnmw |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.21:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-717c3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-717c3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-717c3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-717c3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-717c3 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-717c3 |
InferenceServiceReady |
InferenceService [success-200-isvc-717c3] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-717c3 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-717c3 |
InferenceServiceReady |
InferenceService [error-404-isvc-717c3] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-717c3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-717c3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-717c3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-717c3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x7) | kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-srpkx |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.23:8080: connect: connection refused |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-717c3 |
ScalingReplicaSet |
Scaled up replica set switch-graph-717c3-b69fd89f from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-717c3-b69fd89f-9mwps |
Pulling |
Pulling image "quay.io/opendatahub/kserve-router@sha256:bcb85ef6ed85b7da131182bb5f673741b67400a2637d62d1dcee3f3c438674f0" | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-717c3-b69fd89f |
SuccessfulCreate |
Created pod: switch-graph-717c3-b69fd89f-9mwps | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-717c3 |
UpdateFailed |
Failed to update status for InferenceGraph "switch-graph-717c3": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "switch-graph-717c3": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
multus |
switch-graph-717c3-b69fd89f-9mwps |
AddedInterface |
Add eth0 [10.132.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-717c3 |
InferenceGraphReady |
InferenceGraph [switch-graph-717c3] is Ready | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-717c3 |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "switch-graph-717c3": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
switch-graph-717c3-b69fd89f-9mwps |
Started |
Started container switch-graph-717c3 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-717c3-b69fd89f-9mwps |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-router@sha256:bcb85ef6ed85b7da131182bb5f673741b67400a2637d62d1dcee3f3c438674f0" in 1.816s (1.816s including waiting). Image size: 218226791 bytes. | |
kserve-ci-e2e-test |
kubelet |
switch-graph-717c3-b69fd89f-9mwps |
Created |
Created container: switch-graph-717c3 | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
InferenceServiceReady |
InferenceService [isvc-xgboost-graph] is Ready | |
kserve-ci-e2e-test |
kubelet |
switch-graph-717c3-b69fd89f-9mwps |
Killing |
Stopping container switch-graph-717c3 | |
kserve-ci-e2e-test |
multus |
error-404-isvc-e92a5-predictor-599bf66b77-hrnbd |
AddedInterface |
Add eth0 [10.134.0.23/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e92a5 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-e92a5": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-e92a5": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e92a5 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-e92a5": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-717c3-predictor-67f5fb9455-8qnmw |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-e92a5-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-e92a5-predictor-599bf66b77 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e92a5-predictor-5cdccb45b8-bnrjn |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-e92a5-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-e92a5-predictor-5cdccb45b8 |
SuccessfulCreate |
Created pod: success-200-isvc-e92a5-predictor-5cdccb45b8-bnrjn | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-717c3-predictor-67f5fb9455-8qnmw |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e92a5-predictor-599bf66b77-hrnbd |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1813" already present on machine | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-e92a5-predictor-599bf66b77 |
SuccessfulCreate |
Created pod: error-404-isvc-e92a5-predictor-599bf66b77-hrnbd | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-e92a5-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-e92a5-predictor-5cdccb45b8 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-717c3-predictor-7db58fb9f4-htnjs |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.20:8643/healthz": dial tcp 10.134.0.20:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-717c3-predictor-7db58fb9f4-htnjs |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-717c3-predictor-7db58fb9f4-htnjs |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e92a5-predictor-5cdccb45b8-bnrjn |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e92a5-predictor-5cdccb45b8-bnrjn |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1813" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-717c3-predictor-67f5fb9455-8qnmw |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.21:8643/healthz": dial tcp 10.134.0.21:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e92a5-predictor-599bf66b77-hrnbd |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e92a5-predictor-599bf66b77-hrnbd |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e92a5-predictor-599bf66b77-hrnbd |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-e92a5-predictor-5cdccb45b8-bnrjn |
AddedInterface |
Add eth0 [10.134.0.22/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e92a5-predictor-599bf66b77-hrnbd |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e92a5-predictor-5cdccb45b8-bnrjn |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e92a5-predictor-5cdccb45b8-bnrjn |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e92a5-predictor-5cdccb45b8-bnrjn |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e92a5-predictor-5cdccb45b8-bnrjn |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e92a5-predictor-599bf66b77-hrnbd |
Created |
Created container: kserve-container | |
| (x9) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-85cb8c5574-2ccq6 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.22:8080: connect: connection refused |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-2] is Ready | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e92a5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e92a5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-1] is Ready | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-717c3-b69fd89f-9mwps |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-e92a5-predictor-5cdccb45b8-bnrjn |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.22:8080: connect: connection refused |
kserve-ci-e2e-test |
replicaset-controller |
model-chainer-5595dc65c9 |
SuccessfulCreate |
Created pod: model-chainer-5595dc65c9-w4vhv | |
kserve-ci-e2e-test |
deployment-controller |
model-chainer |
ScalingReplicaSet |
Scaled up replica set model-chainer-5595dc65c9 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
model-chainer |
InferenceGraphReady |
InferenceGraph [model-chainer] is Ready | |
kserve-ci-e2e-test |
kubelet |
model-chainer-5595dc65c9-w4vhv |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "model-chainer-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
model-chainer-5595dc65c9-w4vhv |
Created |
Created container: model-chainer | |
kserve-ci-e2e-test |
kubelet |
model-chainer-5595dc65c9-w4vhv |
Started |
Started container model-chainer | |
kserve-ci-e2e-test |
multus |
model-chainer-5595dc65c9-w4vhv |
AddedInterface |
Add eth0 [10.132.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
model-chainer-5595dc65c9-w4vhv |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:bcb85ef6ed85b7da131182bb5f673741b67400a2637d62d1dcee3f3c438674f0" already present on machine | |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e92a5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e92a5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x7) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x7) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e92a5 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e92a5 |
InferenceServiceReady |
InferenceService [error-404-isvc-e92a5] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e92a5 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e92a5 |
InferenceServiceReady |
InferenceService [success-200-isvc-e92a5] is Ready | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-85cb8c5574-2ccq6 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-srpkx |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-0b9d4-predictor-7fdbfdf779 |
SuccessfulCreate |
Created pod: error-404-isvc-0b9d4-predictor-7fdbfdf779-f5hq5 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-0b9d4-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-0b9d4-predictor-7fdbfdf779 from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-0b9d4 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-0b9d4": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-0b9d4": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-0b9d4 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-0b9d4": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
model-chainer-5595dc65c9-w4vhv |
Killing |
Stopping container model-chainer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-85cb8c5574-2ccq6 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-srpkx |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5bcddb87c8-46rgb |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5bcddb87c8-46rgb |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0b9d4-predictor-54b67cb57d-48n99 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-0b9d4-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-0b9d4-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-0b9d4-predictor-54b67cb57d from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-0b9d4-predictor-54b67cb57d |
SuccessfulCreate |
Created pod: success-200-isvc-0b9d4-predictor-54b67cb57d-48n99 | |
| (x2) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-0b9d4-predictor-7fdbfdf779-f5hq5 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-0b9d4-predictor-serving-cert" not found |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0b9d4-predictor-54b67cb57d-48n99 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0b9d4-predictor-54b67cb57d-48n99 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0b9d4-predictor-54b67cb57d-48n99 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0b9d4-predictor-54b67cb57d-48n99 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0b9d4-predictor-54b67cb57d-48n99 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0b9d4-predictor-54b67cb57d-48n99 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1813" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-0b9d4-predictor-54b67cb57d-48n99 |
AddedInterface |
Add eth0 [10.134.0.24/23] from ovn-kubernetes | |
| (x10) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5bcddb87c8-46rgb |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.33:8080: connect: connection refused |
kserve-ci-e2e-test |
multus |
error-404-isvc-0b9d4-predictor-7fdbfdf779-f5hq5 |
AddedInterface |
Add eth0 [10.134.0.25/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0b9d4-predictor-7fdbfdf779-f5hq5 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0b9d4-predictor-7fdbfdf779-f5hq5 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0b9d4-predictor-7fdbfdf779-f5hq5 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-5bcddb87c8-46rgb |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.33:8643/healthz": dial tcp 10.132.0.33:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0b9d4-predictor-7fdbfdf779-f5hq5 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0b9d4-predictor-7fdbfdf779-f5hq5 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0b9d4-predictor-7fdbfdf779-f5hq5 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1813" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-85cb8c5574-2ccq6 |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.22:8643/healthz": dial tcp 10.133.0.22:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-srpkx |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.23:8643/healthz": dial tcp 10.133.0.23:8643: connect: connection refused | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-e92a5 |
ScalingReplicaSet |
Scaled up replica set switch-graph-e92a5-6d7f7969cc from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-e92a5-6d7f7969cc-8q5zh |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-e92a5-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-e92a5-6d7f7969cc |
SuccessfulCreate |
Created pod: switch-graph-e92a5-6d7f7969cc-8q5zh | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-e92a5 |
InferenceGraphReady |
InferenceGraph [switch-graph-e92a5] is Ready | |
kserve-ci-e2e-test |
multus |
switch-graph-e92a5-6d7f7969cc-8q5zh |
AddedInterface |
Add eth0 [10.132.0.36/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-e92a5-6d7f7969cc-8q5zh |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:bcb85ef6ed85b7da131182bb5f673741b67400a2637d62d1dcee3f3c438674f0" already present on machine | |
kserve-ci-e2e-test |
kubelet |
switch-graph-e92a5-6d7f7969cc-8q5zh |
Created |
Created container: switch-graph-e92a5 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-e92a5-6d7f7969cc-8q5zh |
Started |
Started container switch-graph-e92a5 | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e92a5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-e92a5 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-e92a5 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e92a5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e92a5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e92a5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
model-chainer-5595dc65c9-w4vhv |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-0b9d4-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-0b9d4-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-0b9d4-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-0b9d4-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-0b9d4-predictor-7fdbfdf779-f5hq5 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.25:8080: connect: connection refused |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-e92a5 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-e92a5 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-0b9d4 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-0b9d4 |
InferenceServiceReady |
InferenceService [success-200-isvc-0b9d4] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-0b9d4 |
InferenceServiceReady |
InferenceService [error-404-isvc-0b9d4] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-0b9d4 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-0b9d4-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-0b9d4-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-0b9d4-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-0b9d4 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-0b9d4-7db5cc7799 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-0b9d4-7db5cc7799 |
SuccessfulCreate |
Created pod: sequence-graph-0b9d4-7db5cc7799-hgsps | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-0b9d4-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-0b9d4 |
InferenceGraphReady |
InferenceGraph [sequence-graph-0b9d4] is Ready | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-0b9d4-7db5cc7799-hgsps |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "sequence-graph-0b9d4-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-0b9d4-7db5cc7799-hgsps |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:bcb85ef6ed85b7da131182bb5f673741b67400a2637d62d1dcee3f3c438674f0" already present on machine | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-0b9d4-7db5cc7799-hgsps |
Started |
Started container sequence-graph-0b9d4 | |
kserve-ci-e2e-test |
multus |
sequence-graph-0b9d4-7db5cc7799-hgsps |
AddedInterface |
Add eth0 [10.132.0.37/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-0b9d4-7db5cc7799-hgsps |
Created |
Created container: sequence-graph-0b9d4 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-0b9d4 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-0b9d4 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-af89d-predictor-6bff89d85-j66sp |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1813" already present on machine | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-af89d-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-af89d-predictor-6bff89d85 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
success-200-isvc-af89d-predictor-6fc7bd7dcb-2989w |
AddedInterface |
Add eth0 [10.134.0.26/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-af89d-predictor-6fc7bd7dcb-2989w |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1813" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-af89d-predictor-6fc7bd7dcb-2989w |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-af89d-predictor-6fc7bd7dcb-2989w |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-af89d-predictor-6fc7bd7dcb-2989w |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e92a5-predictor-5cdccb45b8-bnrjn |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
switch-graph-e92a5-6d7f7969cc-8q5zh |
Killing |
Stopping container switch-graph-e92a5 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-af89d-predictor-6fc7bd7dcb-2989w |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-af89d-predictor-6fc7bd7dcb-2989w |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e92a5-predictor-599bf66b77-hrnbd |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-af89d-predictor-6fc7bd7dcb |
SuccessfulCreate |
Created pod: success-200-isvc-af89d-predictor-6fc7bd7dcb-2989w | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-af89d-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-af89d-predictor-6fc7bd7dcb from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-af89d-predictor-6bff89d85 |
SuccessfulCreate |
Created pod: error-404-isvc-af89d-predictor-6bff89d85-j66sp | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e92a5-predictor-599bf66b77-hrnbd |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-af89d-predictor-6bff89d85-j66sp |
AddedInterface |
Add eth0 [10.134.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-af89d-predictor-6bff89d85-j66sp |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-af89d-predictor-6bff89d85-j66sp |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-af89d-predictor-6bff89d85-j66sp |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e92a5-predictor-5cdccb45b8-bnrjn |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-af89d-predictor-6bff89d85-j66sp |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-af89d-predictor-6bff89d85-j66sp |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e92a5-predictor-599bf66b77-hrnbd |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.23:8643/healthz": dial tcp 10.134.0.23:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-e92a5-predictor-599bf66b77-hrnbd |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.23:8080: connect: connection refused |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-af89d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-af89d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-e92a5-6d7f7969cc-8q5zh |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-af89d-predictor-6bff89d85-j66sp |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.27:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-af89d-predictor-6fc7bd7dcb-2989w |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.26:8080: connect: connection refused |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-af89d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-af89d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-af89d |
InferenceServiceReady |
InferenceService [error-404-isvc-af89d] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-af89d |
InferenceServiceReady |
InferenceService [success-200-isvc-af89d] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-af89d |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-af89d |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-222a3-predictor-bb95bb5f7-jbnk4 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-222a3-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0b9d4-predictor-7fdbfdf779-f5hq5 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-222a3 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-222a3": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-222a3-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-222a3-predictor-6d5d87c54d from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-0b9d4-7db5cc7799-hgsps |
Killing |
Stopping container sequence-graph-0b9d4 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0b9d4-predictor-7fdbfdf779-f5hq5 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-222a3-predictor-6d5d87c54d-2tg26 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-222a3-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-222a3 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-222a3": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-222a3": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-222a3-predictor-bb95bb5f7 |
SuccessfulCreate |
Created pod: error-404-isvc-222a3-predictor-bb95bb5f7-jbnk4 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-222a3-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-222a3-predictor-bb95bb5f7 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0b9d4-predictor-54b67cb57d-48n99 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-222a3-predictor-6d5d87c54d |
SuccessfulCreate |
Created pod: success-200-isvc-222a3-predictor-6d5d87c54d-2tg26 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-222a3 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-222a3": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-222a3 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-222a3": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-222a3": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0b9d4-predictor-54b67cb57d-48n99 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-222a3-predictor-6d5d87c54d-2tg26 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-222a3-predictor-bb95bb5f7-jbnk4 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-222a3-predictor-bb95bb5f7-jbnk4 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-222a3-predictor-bb95bb5f7-jbnk4 |
AddedInterface |
Add eth0 [10.134.0.29/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-222a3-predictor-bb95bb5f7-jbnk4 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-222a3-predictor-6d5d87c54d-2tg26 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-222a3-predictor-6d5d87c54d-2tg26 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-222a3-predictor-bb95bb5f7-jbnk4 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-222a3-predictor-bb95bb5f7-jbnk4 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1813" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-222a3-predictor-6d5d87c54d-2tg26 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-222a3-predictor-6d5d87c54d-2tg26 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-222a3-predictor-6d5d87c54d-2tg26 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1813" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-222a3-predictor-6d5d87c54d-2tg26 |
AddedInterface |
Add eth0 [10.134.0.28/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-222a3-predictor-bb95bb5f7-jbnk4 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0b9d4-predictor-54b67cb57d-48n99 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.24:8643/healthz": dial tcp 10.134.0.24:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-0b9d4-predictor-54b67cb57d-48n99 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.24:8080: connect: connection refused |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-af89d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-af89d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-af89d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-af89d |
InferenceGraphReady |
InferenceGraph [ensemble-graph-af89d] is Ready | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-af89d-db7874558-kldsc |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:bcb85ef6ed85b7da131182bb5f673741b67400a2637d62d1dcee3f3c438674f0" already present on machine | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-af89d-db7874558-kldsc |
Created |
Created container: ensemble-graph-af89d | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-af89d |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-af89d-db7874558 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-af89d-db7874558 |
SuccessfulCreate |
Created pod: ensemble-graph-af89d-db7874558-kldsc | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-af89d-db7874558-kldsc |
Started |
Started container ensemble-graph-af89d | |
kserve-ci-e2e-test |
multus |
ensemble-graph-af89d-db7874558-kldsc |
AddedInterface |
Add eth0 [10.132.0.38/23] from ovn-kubernetes | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-af89d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-4e070-predictor-6985d68b68 |
SuccessfulCreate |
Created pod: success-200-isvc-4e070-predictor-6985d68b68-kls5q | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-4e070 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-4e070": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-af89d-db7874558-kldsc |
Killing |
Stopping container ensemble-graph-af89d | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-4e070 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-4e070": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-4e070": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-af89d-predictor-6fc7bd7dcb-2989w |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4e070-predictor-6985d68b68-kls5q |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-4e070-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-af89d-predictor-6fc7bd7dcb-2989w |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-4e070-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-4e070-predictor-6985d68b68 from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-4e070-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-4e070-predictor-f797d89df from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-af89d-predictor-6bff89d85-j66sp |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-af89d-predictor-6bff89d85-j66sp |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-4e070-predictor-f797d89df |
SuccessfulCreate |
Created pod: error-404-isvc-4e070-predictor-f797d89df-t5768 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4e070-predictor-6985d68b68-kls5q |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1813" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4e070-predictor-6985d68b68-kls5q |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4e070-predictor-f797d89df-t5768 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4e070-predictor-f797d89df-t5768 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4e070-predictor-f797d89df-t5768 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4e070-predictor-f797d89df-t5768 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1813" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-4e070-predictor-6985d68b68-kls5q |
AddedInterface |
Add eth0 [10.134.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4e070-predictor-6985d68b68-kls5q |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4e070-predictor-6985d68b68-kls5q |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-4e070-predictor-f797d89df-t5768 |
AddedInterface |
Add eth0 [10.134.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4e070-predictor-6985d68b68-kls5q |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4e070-predictor-f797d89df-t5768 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4e070-predictor-6985d68b68-kls5q |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4e070-predictor-f797d89df-t5768 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-af89d-predictor-6fc7bd7dcb-2989w |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.26:8643/healthz": dial tcp 10.134.0.26:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-af89d-predictor-6bff89d85-j66sp |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.27:8643/healthz": dial tcp 10.134.0.27:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-0b9d4-7db5cc7799-hgsps |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-222a3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-222a3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-222a3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-222a3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-222a3-predictor-bb95bb5f7-jbnk4 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.29:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-af89d-db7874558-kldsc |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-222a3-predictor-6d5d87c54d-2tg26 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.28:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-222a3 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-222a3 |
InferenceServiceReady |
InferenceService [error-404-isvc-222a3] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4e070-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4e070-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-222a3 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-222a3 |
InferenceServiceReady |
InferenceService [success-200-isvc-222a3] is Ready | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-222a3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-222a3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4e070-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4e070-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-4e070 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-4e070 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-4e070 |
InferenceServiceReady |
InferenceService [success-200-isvc-4e070] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-4e070 |
InferenceServiceReady |
InferenceService [error-404-isvc-4e070] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-222a3 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-222a3-5f5974b4b4 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-222a3 |
InferenceGraphReady |
InferenceGraph [sequence-graph-222a3] is Ready | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-222a3-5f5974b4b4-bb4dl |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "sequence-graph-222a3-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-222a3-5f5974b4b4 |
SuccessfulCreate |
Created pod: sequence-graph-222a3-5f5974b4b4-bb4dl | |
kserve-ci-e2e-test |
multus |
sequence-graph-222a3-5f5974b4b4-bb4dl |
AddedInterface |
Add eth0 [10.132.0.39/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-222a3-5f5974b4b4-bb4dl |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:bcb85ef6ed85b7da131182bb5f673741b67400a2637d62d1dcee3f3c438674f0" already present on machine | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-222a3-5f5974b4b4-bb4dl |
Created |
Created container: sequence-graph-222a3 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-222a3-5f5974b4b4-bb4dl |
Started |
Started container sequence-graph-222a3 | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-222a3-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-222a3-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
sequence-graph-222a3-5f5974b4b4-bb4dl |
Killing |
Stopping container sequence-graph-222a3 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-222a3-predictor-6d5d87c54d-2tg26 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-222a3-predictor-bb95bb5f7-jbnk4 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8d409-predictor-5fcc67fb8c-l8z5j |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-8d409-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-8d409-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-8d409-predictor-5fcc67fb8c from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-8d409-predictor-5fcc67fb8c |
SuccessfulCreate |
Created pod: error-404-isvc-8d409-predictor-5fcc67fb8c-l8z5j | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-8d409-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-8d409-predictor-5f6cb868f4 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-4e070-57b477589f |
SuccessfulCreate |
Created pod: ensemble-graph-4e070-57b477589f-dnp8r | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-8d409-predictor-5f6cb868f4 |
SuccessfulCreate |
Created pod: success-200-isvc-8d409-predictor-5f6cb868f4-8njwq | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-8d409 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-8d409": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-8d409": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-222a3-predictor-bb95bb5f7-jbnk4 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4e070-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4e070-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-4e070 |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-4e070-57b477589f from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-4e070 |
InferenceGraphReady |
InferenceGraph [ensemble-graph-4e070] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-8d409 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-8d409": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8d409-predictor-5f6cb868f4-8njwq |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-8d409-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-8d409 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-8d409": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-8d409": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-8d409 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-8d409": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-222a3-predictor-6d5d87c54d-2tg26 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8d409-predictor-5f6cb868f4-8njwq |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8d409-predictor-5fcc67fb8c-l8z5j |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8d409-predictor-5f6cb868f4-8njwq |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1813" already present on machine | |
kserve-ci-e2e-test |
multus |
ensemble-graph-4e070-57b477589f-dnp8r |
AddedInterface |
Add eth0 [10.132.0.40/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
error-404-isvc-8d409-predictor-5fcc67fb8c-l8z5j |
AddedInterface |
Add eth0 [10.134.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-4e070-57b477589f-dnp8r |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:bcb85ef6ed85b7da131182bb5f673741b67400a2637d62d1dcee3f3c438674f0" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8d409-predictor-5fcc67fb8c-l8z5j |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1813" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8d409-predictor-5fcc67fb8c-l8z5j |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8d409-predictor-5fcc67fb8c-l8z5j |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8d409-predictor-5f6cb868f4-8njwq |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-4e070-57b477589f-dnp8r |
Started |
Started container ensemble-graph-4e070 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8d409-predictor-5f6cb868f4-8njwq |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8d409-predictor-5fcc67fb8c-l8z5j |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8d409-predictor-5fcc67fb8c-l8z5j |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-8d409-predictor-5f6cb868f4-8njwq |
AddedInterface |
Add eth0 [10.134.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-4e070-57b477589f-dnp8r |
Created |
Created container: ensemble-graph-4e070 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8d409-predictor-5f6cb868f4-8njwq |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8d409-predictor-5f6cb868f4-8njwq |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-222a3-predictor-bb95bb5f7-jbnk4 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.29:8643/healthz": dial tcp 10.134.0.29:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-222a3-predictor-6d5d87c54d-2tg26 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.28:8643/healthz": dial tcp 10.134.0.28:8643: connect: connection refused | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-4e070 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4e070-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4e070-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-4e070 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-222a3-5f5974b4b4-bb4dl |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8d409-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8d409-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8d409-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-4e070 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-4e070 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8d409-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-8d409-predictor-5f6cb868f4-8njwq |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.32:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-8d409 |
InferenceServiceReady |
InferenceService [success-200-isvc-8d409] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-8d409 |
InferenceServiceReady |
InferenceService [error-404-isvc-8d409] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-8d409 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-8d409 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-8d409 |
InferenceGraphReady |
InferenceGraph [sequence-graph-8d409] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8d409-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8d409-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-8d409 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-8d409-85cbb7d94 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-8d409-85cbb7d94 |
SuccessfulCreate |
Created pod: sequence-graph-8d409-85cbb7d94-dp7l8 | |
| (x2) | kserve-ci-e2e-test |
kubelet |
sequence-graph-8d409-85cbb7d94-dp7l8 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "sequence-graph-8d409-serving-cert" not found |
kserve-ci-e2e-test |
kubelet |
sequence-graph-8d409-85cbb7d94-dp7l8 |
Created |
Created container: sequence-graph-8d409 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-8d409-85cbb7d94-dp7l8 |
Started |
Started container sequence-graph-8d409 | |
kserve-ci-e2e-test |
multus |
sequence-graph-8d409-85cbb7d94-dp7l8 |
AddedInterface |
Add eth0 [10.132.0.41/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-8d409-85cbb7d94-dp7l8 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:bcb85ef6ed85b7da131182bb5f673741b67400a2637d62d1dcee3f3c438674f0" already present on machine | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8d409-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8d409-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-8d409 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-8d409 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-e803a-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-e803a-predictor-76bd58685f from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4e070-predictor-f797d89df-t5768 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4e070-predictor-f797d89df-t5768 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-e803a-predictor-658895f866 |
SuccessfulCreate |
Created pod: success-200-isvc-e803a-predictor-658895f866-4669k | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-e803a-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-e803a-predictor-658895f866 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e803a-predictor-658895f866-4669k |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-e803a-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4e070-predictor-6985d68b68-kls5q |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-e803a-predictor-76bd58685f |
SuccessfulCreate |
Created pod: error-404-isvc-e803a-predictor-76bd58685f-h5jzr | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4e070-predictor-6985d68b68-kls5q |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e803a |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-e803a": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-e803a": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e803a |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-e803a": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e803a-predictor-76bd58685f-h5jzr |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-e803a-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-4e070-57b477589f-dnp8r |
Killing |
Stopping container ensemble-graph-4e070 | |
kserve-ci-e2e-test |
multus |
error-404-isvc-e803a-predictor-76bd58685f-h5jzr |
AddedInterface |
Add eth0 [10.134.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e803a-predictor-658895f866-4669k |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1813" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e803a-predictor-76bd58685f-h5jzr |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e803a-predictor-76bd58685f-h5jzr |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1813" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-e803a-predictor-658895f866-4669k |
AddedInterface |
Add eth0 [10.134.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e803a-predictor-76bd58685f-h5jzr |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e803a-predictor-76bd58685f-h5jzr |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e803a-predictor-76bd58685f-h5jzr |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e803a-predictor-76bd58685f-h5jzr |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e803a-predictor-658895f866-4669k |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e803a-predictor-658895f866-4669k |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e803a-predictor-658895f866-4669k |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e803a-predictor-658895f866-4669k |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e803a-predictor-658895f866-4669k |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4e070-predictor-6985d68b68-kls5q |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.30:8643/healthz": dial tcp 10.134.0.30:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-4e070-predictor-f797d89df-t5768 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.31:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-4e070-predictor-6985d68b68-kls5q |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.30:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4e070-predictor-f797d89df-t5768 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.31:8643/healthz": dial tcp 10.134.0.31:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-4e070-57b477589f-dnp8r |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e803a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e803a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e803a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e803a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-e803a-predictor-76bd58685f-h5jzr |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.35:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-e803a-predictor-658895f866-4669k |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.34:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e803a |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e803a |
InferenceServiceReady |
InferenceService [success-200-isvc-e803a] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e803a |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e803a |
InferenceServiceReady |
InferenceService [error-404-isvc-e803a] is Ready | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a647d-predictor-5474579676-s27ps |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1813" already present on machine | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-a647d-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-a647d-predictor-784bd7c46f from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-a647d |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-a647d": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8d409-predictor-5fcc67fb8c-l8z5j |
Killing |
Stopping container kserve-container | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e803a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e803a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8d409-predictor-5f6cb868f4-8njwq |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8d409-predictor-5f6cb868f4-8njwq |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-e803a |
InferenceGraphReady |
InferenceGraph [splitter-graph-e803a] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e803a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8d409-predictor-5fcc67fb8c-l8z5j |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-a647d-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-a647d-predictor-5474579676 from 0 to 1 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e803a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
sequence-graph-8d409-85cbb7d94-dp7l8 |
Killing |
Stopping container sequence-graph-8d409 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-a647d-predictor-5474579676 |
SuccessfulCreate |
Created pod: success-200-isvc-a647d-predictor-5474579676-s27ps | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a647d-predictor-784bd7c46f-s7brw |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-a647d-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-e803a |
ScalingReplicaSet |
Scaled up replica set splitter-graph-e803a-55fbf78854 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-e803a-55fbf78854 |
SuccessfulCreate |
Created pod: splitter-graph-e803a-55fbf78854-lnmxf | |
kserve-ci-e2e-test |
multus |
success-200-isvc-a647d-predictor-5474579676-s27ps |
AddedInterface |
Add eth0 [10.134.0.36/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a647d-predictor-5474579676-s27ps |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a647d-predictor-5474579676-s27ps |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a647d-predictor-5474579676-s27ps |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a647d-predictor-5474579676-s27ps |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a647d-predictor-5474579676-s27ps |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-a647d |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-a647d": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-a647d": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-a647d-predictor-784bd7c46f |
SuccessfulCreate |
Created pod: error-404-isvc-a647d-predictor-784bd7c46f-s7brw | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a647d-predictor-784bd7c46f-s7brw |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-a647d-predictor-784bd7c46f-s7brw |
AddedInterface |
Add eth0 [10.134.0.37/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a647d-predictor-784bd7c46f-s7brw |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1813" already present on machine | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-e803a-55fbf78854-lnmxf |
Created |
Created container: splitter-graph-e803a | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a647d-predictor-784bd7c46f-s7brw |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a647d-predictor-784bd7c46f-s7brw |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-e803a-55fbf78854-lnmxf |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:bcb85ef6ed85b7da131182bb5f673741b67400a2637d62d1dcee3f3c438674f0" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a647d-predictor-784bd7c46f-s7brw |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-e803a-55fbf78854-lnmxf |
Started |
Started container splitter-graph-e803a | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a647d-predictor-784bd7c46f-s7brw |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
multus |
splitter-graph-e803a-55fbf78854-lnmxf |
AddedInterface |
Add eth0 [10.132.0.42/23] from ovn-kubernetes | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-8d409-predictor-5fcc67fb8c-l8z5j |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.33:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8d409-predictor-5fcc67fb8c-l8z5j |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.33:8643/healthz": dial tcp 10.134.0.33:8643: connect: connection refused | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f7711 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-f7711": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f7711 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-f7711": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-f7711": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-f7711-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-f7711-predictor-7b6f54fcd8 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-f7711-predictor-6d447c4c9c |
SuccessfulCreate |
Created pod: error-404-isvc-f7711-predictor-6d447c4c9c-vwjvk | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e803a-predictor-76bd58685f-h5jzr |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e803a-predictor-76bd58685f-h5jzr |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e803a-predictor-658895f866-4669k |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e803a-predictor-658895f866-4669k |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-e803a-55fbf78854-lnmxf |
Killing |
Stopping container splitter-graph-e803a | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-f7711-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-f7711-predictor-6d447c4c9c from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-f7711-predictor-7b6f54fcd8 |
SuccessfulCreate |
Created pod: success-200-isvc-f7711-predictor-7b6f54fcd8-dkzjv | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f7711-predictor-7b6f54fcd8-dkzjv |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-f7711-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f7711-predictor-7b6f54fcd8-dkzjv |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
multus |
success-200-isvc-f7711-predictor-7b6f54fcd8-dkzjv |
AddedInterface |
Add eth0 [10.134.0.38/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f7711-predictor-7b6f54fcd8-dkzjv |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1813" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f7711-predictor-7b6f54fcd8-dkzjv |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f7711-predictor-7b6f54fcd8-dkzjv |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
| (x2) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-f7711-predictor-6d447c4c9c-vwjvk |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-f7711-predictor-serving-cert" not found |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f7711-predictor-7b6f54fcd8-dkzjv |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f7711-predictor-7b6f54fcd8-dkzjv |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f7711-predictor-6d447c4c9c-vwjvk |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-f7711-predictor-6d447c4c9c-vwjvk |
AddedInterface |
Add eth0 [10.134.0.39/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f7711-predictor-6d447c4c9c-vwjvk |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1813" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f7711-predictor-6d447c4c9c-vwjvk |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f7711-predictor-6d447c4c9c-vwjvk |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f7711-predictor-6d447c4c9c-vwjvk |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f7711-predictor-6d447c4c9c-vwjvk |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e803a-predictor-658895f866-4669k |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.34:8643/healthz": dial tcp 10.134.0.34:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e803a-predictor-76bd58685f-h5jzr |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.35:8643/healthz": dial tcp 10.134.0.35:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-8d409-85cbb7d94-dp7l8 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-a647d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-a647d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-a647d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-a647d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-a647d-predictor-784bd7c46f-s7brw |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.37:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-e803a-55fbf78854-lnmxf |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-a647d-predictor-5474579676-s27ps |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.36:8080: connect: connection refused |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f7711-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f7711-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f7711-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f7711-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-a647d |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-a647d |
InferenceServiceReady |
InferenceService [error-404-isvc-a647d] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-a647d |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-a647d |
InferenceServiceReady |
InferenceService [success-200-isvc-a647d] is Ready | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-f7711-predictor-7b6f54fcd8-dkzjv |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.38:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-f7711-predictor-6d447c4c9c-vwjvk |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.39:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-f7711 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-f7711 |
InferenceServiceReady |
InferenceService [success-200-isvc-f7711] is Ready | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-a647d |
InferenceGraphReady |
InferenceGraph [switch-graph-a647d] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f7711 |
InferenceServiceReady |
InferenceService [error-404-isvc-f7711] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-a647d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-a647d |
ScalingReplicaSet |
Scaled up replica set switch-graph-a647d-7bc89888f9 from 0 to 1 | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f7711 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-a647d-7bc89888f9 |
SuccessfulCreate |
Created pod: switch-graph-a647d-7bc89888f9-ftrq7 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-a647d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
switch-graph-a647d-7bc89888f9-ftrq7 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-a647d-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
switch-graph-a647d-7bc89888f9-ftrq7 |
Created |
Created container: switch-graph-a647d | |
kserve-ci-e2e-test |
kubelet |
switch-graph-a647d-7bc89888f9-ftrq7 |
Started |
Started container switch-graph-a647d | |
kserve-ci-e2e-test |
multus |
switch-graph-a647d-7bc89888f9-ftrq7 |
AddedInterface |
Add eth0 [10.132.0.43/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-a647d-7bc89888f9-ftrq7 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:bcb85ef6ed85b7da131182bb5f673741b67400a2637d62d1dcee3f3c438674f0" already present on machine | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f7711-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-f7711 |
InferenceGraphReady |
InferenceGraph [splitter-graph-f7711] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-f7711 |
ScalingReplicaSet |
Scaled up replica set splitter-graph-f7711-5d9485b744 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-f7711-5d9485b744 |
SuccessfulCreate |
Created pod: splitter-graph-f7711-5d9485b744-t55xb | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f7711-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
splitter-graph-f7711-5d9485b744-t55xb |
Created |
Created container: splitter-graph-f7711 | |
kserve-ci-e2e-test |
multus |
splitter-graph-f7711-5d9485b744-t55xb |
AddedInterface |
Add eth0 [10.132.0.44/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-f7711-5d9485b744-t55xb |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:bcb85ef6ed85b7da131182bb5f673741b67400a2637d62d1dcee3f3c438674f0" already present on machine | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-f7711-5d9485b744-t55xb |
Started |
Started container splitter-graph-f7711 | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-a647d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-a647d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-a647d |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-a647d |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f7711-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f7711-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-a647d |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-a647d |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-f7711 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-f7711 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f7711-predictor-7b6f54fcd8-dkzjv |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f7711-predictor-6d447c4c9c-vwjvk |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-f7711-5d9485b744-t55xb |
Killing |
Stopping container splitter-graph-f7711 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f7711-predictor-6d447c4c9c-vwjvk |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f7711-predictor-7b6f54fcd8-dkzjv |
Killing |
Stopping container kube-rbac-proxy | |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-f7711-5d9485b744-t55xb |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
switch-graph-a647d-7bc89888f9-ftrq7 |
Killing |
Stopping container switch-graph-a647d | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a647d-predictor-784bd7c46f-s7brw |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a647d-predictor-784bd7c46f-s7brw |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a647d-predictor-5474579676-s27ps |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-a647d-predictor-5474579676-s27ps |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-a647d-predictor-784bd7c46f-s7brw |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.37:8643/healthz": dial tcp 10.134.0.37:8643: connect: connection refused | |
| (x5) | kserve-ci-e2e-test |
kubelet |
switch-graph-a647d-7bc89888f9-ftrq7 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |