| Time | Namespace | Component | RelatedObject | Reason | Message |
|---|---|---|---|---|---|
kserve-ci-e2e-test |
success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx to ip-10-0-135-193.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-4a77e-predictor-66c6b8db58-dcpsh |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-4a77e-predictor-66c6b8db58-dcpsh to ip-10-0-133-113.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7 to ip-10-0-135-193.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-2aa9e-744c775558-zn5kc |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc to ip-10-0-135-193.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-b0976-predictor-65dc4f7b7d-4n7pq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-b0976-predictor-65dc4f7b7d-4n7pq to ip-10-0-133-113.ec2.internal | ||
kserve-ci-e2e-test |
model-chainer-f7558cd98-v8lf8 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8 to ip-10-0-135-193.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-2-predictor-57f7556d97-gb7xp |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-57f7556d97-gb7xp to ip-10-0-135-240.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-fb34c-predictor-67b8546c54-4wjz5 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-fb34c-predictor-67b8546c54-4wjz5 to ip-10-0-133-113.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-2aa9e-predictor-7ff7b8dffd-82f85 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-2aa9e-predictor-7ff7b8dffd-82f85 to ip-10-0-133-113.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm to ip-10-0-135-193.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-04b70-6564cd7dcf-j6jhv |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv to ip-10-0-135-193.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-4a77e-7897f464d9-9scnp |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp to ip-10-0-135-193.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-fb34c-7887db4c77-7fjcb |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb to ip-10-0-135-193.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-70b74-69b5767c67-nxls7 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7 to ip-10-0-135-193.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-b0976-8d4c9ffcc-vg6pl |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl to ip-10-0-135-193.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-8365d-predictor-7567c8f6b-jgvv5 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-8365d-predictor-7567c8f6b-jgvv5 to ip-10-0-133-113.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-1a268-predictor-58cf6c8694-2nfg5 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-1a268-predictor-58cf6c8694-2nfg5 to ip-10-0-133-113.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s to ip-10-0-135-193.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm to ip-10-0-135-193.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-0c0cc-predictor-55555bf8bd-d9fb6 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-0c0cc-predictor-55555bf8bd-d9fb6 to ip-10-0-133-113.ec2.internal | ||
kserve-ci-e2e-test |
isvc-xgboost-graph-predictor-555fc69d56-t74n7 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-t74n7 to ip-10-0-135-240.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-e0a3a-predictor-64d96fc94-zpgcc |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-e0a3a-predictor-64d96fc94-zpgcc to ip-10-0-133-113.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-70b74-predictor-68dbc46586-btrs2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-70b74-predictor-68dbc46586-btrs2 to ip-10-0-133-113.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-1-predictor-8587dd57d4-hmdx7 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-8587dd57d4-hmdx7 to ip-10-0-135-240.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s to ip-10-0-135-193.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-04b70-predictor-5cff8969d4-8k7db |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-04b70-predictor-5cff8969d4-8k7db to ip-10-0-133-113.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-04b70-predictor-697dcfdbb9-k58nf |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf to ip-10-0-135-193.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-0c0cc-5c697ddcd5-bzdsc |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc to ip-10-0-135-193.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf to ip-10-0-135-193.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg to ip-10-0-135-193.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-8365d-79d674c554-6js2m |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m to ip-10-0-135-193.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-1a268-7579f4dc47-xgz4s |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s to ip-10-0-135-193.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-e0a3a-6b8c749bb4-pbx25 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25 to ip-10-0-135-193.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-8365d-predictor-69558d5f59-868nq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq to ip-10-0-135-193.ec2.internal | ||
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-70b74-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-70b74-predictor-5d6ff7f77f from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-70b74-predictor-5d6ff7f77f |
SuccessfulCreate |
Created pod: success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7 | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-8587dd57d4-hmdx7 |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-2-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-2-predictor-57f7556d97 from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-1-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-1-predictor-8587dd57d4 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7 |
Pulling |
Pulling image "quay.io/opendatahub/success-200-isvc:odh-pr-1919" | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-1-predictor-8587dd57d4-hmdx7 |
AddedInterface |
Add eth0 [10.134.0.25/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-8587dd57d4-hmdx7 |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:cd807572e76a5c04b5e09287f537df3a5a34e8aea047c82cf332b6328ae06423" already present on machine | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-70b74-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-70b74-predictor-68dbc46586 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-1-predictor-8587dd57d4 |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-1-predictor-8587dd57d4-hmdx7 | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-2-predictor-57f7556d97 |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-2-predictor-57f7556d97-gb7xp | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-8587dd57d4-hmdx7 |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
multus |
success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7 |
AddedInterface |
Add eth0 [10.132.0.16/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-t74n7 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "isvc-xgboost-graph-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-70b74-predictor-68dbc46586 |
SuccessfulCreate |
Created pod: error-404-isvc-70b74-predictor-68dbc46586-btrs2 | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-xgboost-graph-predictor-555fc69d56 |
SuccessfulCreate |
Created pod: isvc-xgboost-graph-predictor-555fc69d56-t74n7 | |
kserve-ci-e2e-test |
deployment-controller |
isvc-xgboost-graph-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-xgboost-graph-predictor-555fc69d56 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-t74n7 |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:cd807572e76a5c04b5e09287f537df3a5a34e8aea047c82cf332b6328ae06423" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-57f7556d97-gb7xp |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-t74n7 |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-57f7556d97-gb7xp |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-57f7556d97-gb7xp |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:cd807572e76a5c04b5e09287f537df3a5a34e8aea047c82cf332b6328ae06423" already present on machine | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-2-predictor-57f7556d97-gb7xp |
AddedInterface |
Add eth0 [10.134.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-t74n7 |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
multus |
isvc-xgboost-graph-predictor-555fc69d56-t74n7 |
AddedInterface |
Add eth0 [10.134.0.26/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-70b74-predictor-68dbc46586-btrs2 |
Pulling |
Pulling image "quay.io/opendatahub/error-404-isvc:odh-pr-1919" | |
kserve-ci-e2e-test |
multus |
error-404-isvc-70b74-predictor-68dbc46586-btrs2 |
AddedInterface |
Add eth0 [10.133.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-t74n7 |
Pulling |
Pulling image "kserve/xgbserver:latest" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-57f7556d97-gb7xp |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1919" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-8587dd57d4-hmdx7 |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1919" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-70b74-predictor-68dbc46586-btrs2 |
Pulled |
Successfully pulled image "quay.io/opendatahub/error-404-isvc:odh-pr-1919" in 11.633s (11.633s including waiting). Image size: 1357375559 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-70b74-predictor-68dbc46586-btrs2 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7 |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7 |
Pulled |
Successfully pulled image "quay.io/opendatahub/success-200-isvc:odh-pr-1919" in 12.231s (12.231s including waiting). Image size: 1356743244 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-70b74-predictor-68dbc46586-btrs2 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-70b74-predictor-68dbc46586-btrs2 |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-70b74-predictor-68dbc46586-btrs2 |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.635s (2.636s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-70b74-predictor-68dbc46586-btrs2 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-70b74-predictor-68dbc46586-btrs2 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7 |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.405s (2.405s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7 |
Started |
Started container kube-rbac-proxy | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-70b74-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-70b74-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-57f7556d97-gb7xp |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1919" in 29.151s (29.151s including waiting). Image size: 1583073169 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-57f7556d97-gb7xp |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-57f7556d97-gb7xp |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-8587dd57d4-hmdx7 |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-t74n7 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-t74n7 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-t74n7 |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-t74n7 |
Pulled |
Successfully pulled image "kserve/xgbserver:latest" in 29.544s (29.544s including waiting). Image size: 1435742979 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-8587dd57d4-hmdx7 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-8587dd57d4-hmdx7 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-8587dd57d4-hmdx7 |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1919" in 29.47s (29.47s including waiting). Image size: 1583073169 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-57f7556d97-gb7xp |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-t74n7 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-t74n7 |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.714s (2.714s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-8587dd57d4-hmdx7 |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.773s (2.773s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-57f7556d97-gb7xp |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.963s (2.963s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-57f7556d97-gb7xp |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-57f7556d97-gb7xp |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-8587dd57d4-hmdx7 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-8587dd57d4-hmdx7 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-t74n7 |
Started |
Started container kube-rbac-proxy | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-70b74-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-70b74-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.16:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-70b74-predictor-68dbc46586-btrs2 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.33:8080: connect: connection refused |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-70b74 |
InferenceServiceReady |
InferenceService [success-200-isvc-70b74] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-70b74 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-70b74 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-70b74 |
InferenceServiceReady |
InferenceService [error-404-isvc-70b74] is Ready | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-70b74-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-70b74-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
switch-graph-70b74-69b5767c67-nxls7 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-70b74-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-70b74 |
ScalingReplicaSet |
Scaled up replica set switch-graph-70b74-69b5767c67 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-70b74 |
InferenceGraphReady |
InferenceGraph [switch-graph-70b74] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-70b74-69b5767c67 |
SuccessfulCreate |
Created pod: switch-graph-70b74-69b5767c67-nxls7 | |
kserve-ci-e2e-test |
multus |
switch-graph-70b74-69b5767c67-nxls7 |
AddedInterface |
Add eth0 [10.132.0.17/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-70b74-69b5767c67-nxls7 |
Pulling |
Pulling image "quay.io/opendatahub/kserve-router@sha256:0c87e1a125451a8a0f1f46fadc66ac5afc05f5cb73608aee9bffb1f827822f57" | |
kserve-ci-e2e-test |
kubelet |
switch-graph-70b74-69b5767c67-nxls7 |
Created |
Created container: switch-graph-70b74 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-70b74-69b5767c67-nxls7 |
Started |
Started container switch-graph-70b74 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-70b74-69b5767c67-nxls7 |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-router@sha256:0c87e1a125451a8a0f1f46fadc66ac5afc05f5cb73608aee9bffb1f827822f57" in 2.296s (2.296s including waiting). Image size: 211751503 bytes. | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-70b74-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-70b74-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
switch-graph-70b74-69b5767c67-nxls7 |
Killing |
Stopping container switch-graph-70b74 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-0c0cc |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-0c0cc": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0c0cc-predictor-55555bf8bd-d9fb6 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-0c0cc-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-0c0cc |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-0c0cc": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-0c0cc": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-0c0cc-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-0c0cc-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-0c0cc-predictor-565c9b6cf7 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-0c0cc-predictor-565c9b6cf7 |
SuccessfulCreate |
Created pod: success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-70b74-predictor-68dbc46586-btrs2 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-70b74-predictor-68dbc46586-btrs2 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-0c0cc-predictor-55555bf8bd |
SuccessfulCreate |
Created pod: error-404-isvc-0c0cc-predictor-55555bf8bd-d9fb6 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-0c0cc-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-0c0cc-predictor-55555bf8bd from 0 to 1 | |
kserve-ci-e2e-test |
multus |
error-404-isvc-0c0cc-predictor-55555bf8bd-d9fb6 |
AddedInterface |
Add eth0 [10.133.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s |
AddedInterface |
Add eth0 [10.132.0.18/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0c0cc-predictor-55555bf8bd-d9fb6 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0c0cc-predictor-55555bf8bd-d9fb6 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0c0cc-predictor-55555bf8bd-d9fb6 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1919" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0c0cc-predictor-55555bf8bd-d9fb6 |
Started |
Started container kserve-container | |
| (x8) | kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-t74n7 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.26:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0c0cc-predictor-55555bf8bd-d9fb6 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1919" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0c0cc-predictor-55555bf8bd-d9fb6 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7 |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.16:8643/healthz": dial tcp 10.132.0.16:8643: connect: connection refused | |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
InferenceServiceReady |
InferenceService [isvc-xgboost-graph] is Ready | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-2] is Ready | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-1] is Ready | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-70b74-69b5767c67-nxls7 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-0c0cc-predictor-55555bf8bd-d9fb6 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.34:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.18:8080: connect: connection refused |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-0c0cc-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
replicaset-controller |
model-chainer-f7558cd98 |
SuccessfulCreate |
Created pod: model-chainer-f7558cd98-v8lf8 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-0c0cc-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
InferenceGraphController |
model-chainer |
InferenceGraphReady |
InferenceGraph [model-chainer] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
model-chainer |
ScalingReplicaSet |
Scaled up replica set model-chainer-f7558cd98 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
model-chainer-f7558cd98-v8lf8 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "model-chainer-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
model-chainer-f7558cd98-v8lf8 |
Created |
Created container: model-chainer | |
kserve-ci-e2e-test |
kubelet |
model-chainer-f7558cd98-v8lf8 |
Started |
Started container model-chainer | |
kserve-ci-e2e-test |
kubelet |
model-chainer-f7558cd98-v8lf8 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:0c87e1a125451a8a0f1f46fadc66ac5afc05f5cb73608aee9bffb1f827822f57" already present on machine | |
kserve-ci-e2e-test |
multus |
model-chainer-f7558cd98-v8lf8 |
AddedInterface |
Add eth0 [10.132.0.19/23] from ovn-kubernetes | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-0c0cc |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-0c0cc |
InferenceServiceReady |
InferenceService [error-404-isvc-0c0cc] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-0c0cc |
InferenceServiceReady |
InferenceService [success-200-isvc-0c0cc] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-0c0cc |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-4a77e-predictor-66c6b8db58 |
SuccessfulCreate |
Created pod: error-404-isvc-4a77e-predictor-66c6b8db58-dcpsh | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-8587dd57d4-hmdx7 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-4a77e-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-4a77e-predictor-66c6b8db58 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-t74n7 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-t74n7 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-8587dd57d4-hmdx7 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-t74n7 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.26:8643/healthz": dial tcp 10.134.0.26:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
model-chainer-f7558cd98-v8lf8 |
Killing |
Stopping container model-chainer | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-4a77e-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-4a77e-predictor-5b6b79bb5f from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-4a77e-predictor-5b6b79bb5f |
SuccessfulCreate |
Created pod: success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-4a77e-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-57f7556d97-gb7xp |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-57f7556d97-gb7xp |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm |
AddedInterface |
Add eth0 [10.132.0.20/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a77e-predictor-66c6b8db58-dcpsh |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-4a77e-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1919" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a77e-predictor-66c6b8db58-dcpsh |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a77e-predictor-66c6b8db58-dcpsh |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a77e-predictor-66c6b8db58-dcpsh |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1919" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-4a77e-predictor-66c6b8db58-dcpsh |
AddedInterface |
Add eth0 [10.133.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a77e-predictor-66c6b8db58-dcpsh |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a77e-predictor-66c6b8db58-dcpsh |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a77e-predictor-66c6b8db58-dcpsh |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
switch-graph-0c0cc-5c697ddcd5-bzdsc |
AddedInterface |
Add eth0 [10.132.0.21/23] from ovn-kubernetes | |
| (x10) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-8587dd57d4-hmdx7 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.25:8080: connect: connection refused |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-0c0cc |
InferenceGraphReady |
InferenceGraph [switch-graph-0c0cc] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-0c0cc |
ScalingReplicaSet |
Scaled up replica set switch-graph-0c0cc-5c697ddcd5 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-0c0cc-5c697ddcd5 |
SuccessfulCreate |
Created pod: switch-graph-0c0cc-5c697ddcd5-bzdsc | |
kserve-ci-e2e-test |
kubelet |
switch-graph-0c0cc-5c697ddcd5-bzdsc |
Started |
Started container switch-graph-0c0cc | |
kserve-ci-e2e-test |
kubelet |
switch-graph-0c0cc-5c697ddcd5-bzdsc |
Created |
Created container: switch-graph-0c0cc | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-0c0cc-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
switch-graph-0c0cc-5c697ddcd5-bzdsc |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:0c87e1a125451a8a0f1f46fadc66ac5afc05f5cb73608aee9bffb1f827822f57" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-57f7556d97-gb7xp |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.27:8643/healthz": dial tcp 10.134.0.27:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-8587dd57d4-hmdx7 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.25:8643/healthz": dial tcp 10.134.0.25:8643: connect: connection refused | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-0c0cc-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x10) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-57f7556d97-gb7xp |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.27:8080: connect: connection refused |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-0c0cc |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-0c0cc-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-0c0cc-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-0c0cc-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-0c0cc-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-0c0cc |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
model-chainer-f7558cd98-v8lf8 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4a77e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4a77e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4a77e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4a77e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-0c0cc |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-0c0cc |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.20:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-4a77e |
InferenceServiceReady |
InferenceService [error-404-isvc-4a77e] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-4a77e |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-4a77e |
InferenceServiceReady |
InferenceService [success-200-isvc-4a77e] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-4a77e |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-4a77e-7897f464d9 |
SuccessfulCreate |
Created pod: sequence-graph-4a77e-7897f464d9-9scnp | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-4a77e |
ScalingReplicaSet |
Scaled up replica set sequence-graph-4a77e-7897f464d9 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-4a77e |
InferenceGraphReady |
InferenceGraph [sequence-graph-4a77e] is Ready | |
kserve-ci-e2e-test |
multus |
sequence-graph-4a77e-7897f464d9-9scnp |
AddedInterface |
Add eth0 [10.132.0.22/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-4a77e-7897f464d9-9scnp |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:0c87e1a125451a8a0f1f46fadc66ac5afc05f5cb73608aee9bffb1f827822f57" already present on machine | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-4a77e-7897f464d9-9scnp |
Created |
Created container: sequence-graph-4a77e | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-4a77e-7897f464d9-9scnp |
Started |
Started container sequence-graph-4a77e | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4a77e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4a77e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-4a77e |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-4a77e |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4a77e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4a77e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-4a77e |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-4a77e |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-b0976-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-b0976-predictor-65dc4f7b7d from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-b0976-predictor-6c4d57ffcf |
SuccessfulCreate |
Created pod: success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm | |
kserve-ci-e2e-test |
kubelet |
switch-graph-0c0cc-5c697ddcd5-bzdsc |
Killing |
Stopping container switch-graph-0c0cc | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0c0cc-predictor-55555bf8bd-d9fb6 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-b0976-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-b0976-predictor-6c4d57ffcf from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0c0cc-predictor-55555bf8bd-d9fb6 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-b0976-predictor-65dc4f7b7d |
SuccessfulCreate |
Created pod: error-404-isvc-b0976-predictor-65dc4f7b7d-4n7pq | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1919" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b0976-predictor-65dc4f7b7d-4n7pq |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-b0976-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm |
AddedInterface |
Add eth0 [10.132.0.23/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b0976-predictor-65dc4f7b7d-4n7pq |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b0976-predictor-65dc4f7b7d-4n7pq |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b0976-predictor-65dc4f7b7d-4n7pq |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-b0976-predictor-65dc4f7b7d-4n7pq |
AddedInterface |
Add eth0 [10.133.0.36/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b0976-predictor-65dc4f7b7d-4n7pq |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1919" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b0976-predictor-65dc4f7b7d-4n7pq |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b0976-predictor-65dc4f7b7d-4n7pq |
Created |
Created container: kserve-container | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-0c0cc-5c697ddcd5-bzdsc |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-b0976-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-b0976-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.23:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-b0976-predictor-65dc4f7b7d-4n7pq |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.36:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-b0976-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-b0976-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-b0976 |
InferenceServiceReady |
InferenceService [success-200-isvc-b0976] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-b0976 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-b0976 |
InferenceServiceReady |
InferenceService [error-404-isvc-b0976] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-b0976 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
kubelet |
sequence-graph-4a77e-7897f464d9-9scnp |
Killing |
Stopping container sequence-graph-4a77e | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-fb34c-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-fb34c-predictor-64f6cd7f8c from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-fb34c-predictor-67b8546c54 |
SuccessfulCreate |
Created pod: error-404-isvc-fb34c-predictor-67b8546c54-4wjz5 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-fb34c-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-fb34c-predictor-67b8546c54 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-fb34c-predictor-67b8546c54-4wjz5 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-fb34c-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-fb34c-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a77e-predictor-66c6b8db58-dcpsh |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a77e-predictor-66c6b8db58-dcpsh |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-fb34c-predictor-64f6cd7f8c |
SuccessfulCreate |
Created pod: success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf | |
kserve-ci-e2e-test |
multus |
error-404-isvc-fb34c-predictor-67b8546c54-4wjz5 |
AddedInterface |
Add eth0 [10.133.0.37/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-fb34c-predictor-67b8546c54-4wjz5 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf |
AddedInterface |
Add eth0 [10.132.0.24/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1919" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-fb34c-predictor-67b8546c54-4wjz5 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-fb34c-predictor-67b8546c54-4wjz5 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-fb34c-predictor-67b8546c54-4wjz5 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-fb34c-predictor-67b8546c54-4wjz5 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1919" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-fb34c-predictor-67b8546c54-4wjz5 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-b0976-8d4c9ffcc |
SuccessfulCreate |
Created pod: ensemble-graph-b0976-8d4c9ffcc-vg6pl | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a77e-predictor-66c6b8db58-dcpsh |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.35:8080: connect: connection refused |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-b0976-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-b0976-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-b0976-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4a77e-predictor-66c6b8db58-dcpsh |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.35:8643/healthz": dial tcp 10.133.0.35:8643: connect: connection refused | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-b0976 |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-b0976-8d4c9ffcc from 0 to 1 | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-b0976-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-b0976 |
InferenceGraphReady |
InferenceGraph [ensemble-graph-b0976] is Ready | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-b0976-8d4c9ffcc-vg6pl |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "ensemble-graph-b0976-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-b0976-8d4c9ffcc-vg6pl |
Started |
Started container ensemble-graph-b0976 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-b0976-8d4c9ffcc-vg6pl |
Created |
Created container: ensemble-graph-b0976 | |
kserve-ci-e2e-test |
multus |
ensemble-graph-b0976-8d4c9ffcc-vg6pl |
AddedInterface |
Add eth0 [10.132.0.25/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-b0976-8d4c9ffcc-vg6pl |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:0c87e1a125451a8a0f1f46fadc66ac5afc05f5cb73608aee9bffb1f827822f57" already present on machine | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-b0976-8d4c9ffcc-vg6pl |
Killing |
Stopping container ensemble-graph-b0976 | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-2aa9e-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-2aa9e-predictor-777dd588ff from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2aa9e-predictor-7ff7b8dffd-82f85 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-2aa9e-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1919" already present on machine | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-2aa9e-predictor-777dd588ff |
SuccessfulCreate |
Created pod: success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx | |
kserve-ci-e2e-test |
multus |
success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx |
AddedInterface |
Add eth0 [10.132.0.26/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b0976-predictor-65dc4f7b7d-4n7pq |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-2aa9e-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-2aa9e-predictor-7ff7b8dffd from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b0976-predictor-65dc4f7b7d-4n7pq |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-2aa9e-predictor-7ff7b8dffd |
SuccessfulCreate |
Created pod: error-404-isvc-2aa9e-predictor-7ff7b8dffd-82f85 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2aa9e-predictor-7ff7b8dffd-82f85 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2aa9e-predictor-7ff7b8dffd-82f85 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2aa9e-predictor-7ff7b8dffd-82f85 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1919" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2aa9e-predictor-7ff7b8dffd-82f85 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2aa9e-predictor-7ff7b8dffd-82f85 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-2aa9e-predictor-7ff7b8dffd-82f85 |
AddedInterface |
Add eth0 [10.133.0.38/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.23:8643/healthz": dial tcp 10.132.0.23:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2aa9e-predictor-7ff7b8dffd-82f85 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b0976-predictor-65dc4f7b7d-4n7pq |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.36:8643/healthz": dial tcp 10.133.0.36:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-4a77e-7897f464d9-9scnp |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-fb34c-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-fb34c-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-fb34c-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-fb34c-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.24:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-fb34c-predictor-67b8546c54-4wjz5 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.37:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-b0976-8d4c9ffcc-vg6pl |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2aa9e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2aa9e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2aa9e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2aa9e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-fb34c |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-fb34c |
InferenceServiceReady |
InferenceService [success-200-isvc-fb34c] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-fb34c |
InferenceServiceReady |
InferenceService [error-404-isvc-fb34c] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-fb34c |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-2aa9e-predictor-7ff7b8dffd-82f85 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.38:8080: connect: connection refused |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-fb34c |
ScalingReplicaSet |
Scaled up replica set sequence-graph-fb34c-7887db4c77 from 0 to 1 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-fb34c-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-fb34c-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-fb34c-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-fb34c-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-fb34c-7887db4c77 |
SuccessfulCreate |
Created pod: sequence-graph-fb34c-7887db4c77-7fjcb | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-fb34c |
InferenceGraphReady |
InferenceGraph [sequence-graph-fb34c] is Ready | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-fb34c-7887db4c77-7fjcb |
Started |
Started container sequence-graph-fb34c | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-fb34c-7887db4c77-7fjcb |
Created |
Created container: sequence-graph-fb34c | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-2aa9e |
InferenceServiceReady |
InferenceService [success-200-isvc-2aa9e] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-2aa9e |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
multus |
sequence-graph-fb34c-7887db4c77-7fjcb |
AddedInterface |
Add eth0 [10.132.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-fb34c-7887db4c77-7fjcb |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:0c87e1a125451a8a0f1f46fadc66ac5afc05f5cb73608aee9bffb1f827822f57" already present on machine | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-2aa9e |
InferenceServiceReady |
InferenceService [error-404-isvc-2aa9e] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-2aa9e |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-fb34c-predictor-67b8546c54-4wjz5 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-fb34c-7887db4c77-7fjcb |
Killing |
Stopping container sequence-graph-fb34c | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-04b70-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-04b70-predictor-697dcfdbb9 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-04b70-predictor-697dcfdbb9 |
SuccessfulCreate |
Created pod: success-200-isvc-04b70-predictor-697dcfdbb9-k58nf | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-04b70-predictor-5cff8969d4 |
SuccessfulCreate |
Created pod: error-404-isvc-04b70-predictor-5cff8969d4-8k7db | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-04b70-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-04b70-predictor-5cff8969d4 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-fb34c-predictor-67b8546c54-4wjz5 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-04b70-predictor-697dcfdbb9-k58nf |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1919" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-04b70-predictor-5cff8969d4-8k7db |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1919" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-04b70-predictor-697dcfdbb9-k58nf |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-04b70-predictor-697dcfdbb9-k58nf |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-04b70-predictor-5cff8969d4-8k7db |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-04b70-predictor-5cff8969d4-8k7db |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-04b70-predictor-5cff8969d4-8k7db |
AddedInterface |
Add eth0 [10.133.0.39/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-04b70-predictor-697dcfdbb9-k58nf |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-04b70-predictor-5cff8969d4-8k7db |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-04b70-predictor-5cff8969d4-8k7db |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-04b70-predictor-697dcfdbb9-k58nf |
AddedInterface |
Add eth0 [10.132.0.28/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-04b70-predictor-5cff8969d4-8k7db |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-04b70-predictor-697dcfdbb9-k58nf |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-04b70-predictor-697dcfdbb9-k58nf |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.24:8643/healthz": dial tcp 10.132.0.24:8643: connect: connection refused | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-2aa9e-744c775558 |
SuccessfulCreate |
Created pod: ensemble-graph-2aa9e-744c775558-zn5kc | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-2aa9e-744c775558-zn5kc |
Started |
Started container ensemble-graph-2aa9e | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-2aa9e-744c775558-zn5kc |
Created |
Created container: ensemble-graph-2aa9e | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-2aa9e |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-2aa9e-744c775558 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-2aa9e |
InferenceGraphReady |
InferenceGraph [ensemble-graph-2aa9e] is Ready | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-2aa9e-744c775558-zn5kc |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:0c87e1a125451a8a0f1f46fadc66ac5afc05f5cb73608aee9bffb1f827822f57" already present on machine | |
kserve-ci-e2e-test |
multus |
ensemble-graph-2aa9e-744c775558-zn5kc |
AddedInterface |
Add eth0 [10.132.0.29/23] from ovn-kubernetes | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2aa9e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2aa9e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-04b70-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-04b70-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2aa9e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-2aa9e |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-2aa9e |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2aa9e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-fb34c-7887db4c77-7fjcb |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-04b70-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-04b70-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-2aa9e |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-2aa9e |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-04b70 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-04b70 |
InferenceServiceReady |
InferenceService [success-200-isvc-04b70] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-04b70 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-04b70 |
InferenceServiceReady |
InferenceService [error-404-isvc-04b70] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-04b70-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-04b70-6564cd7dcf |
SuccessfulCreate |
Created pod: sequence-graph-04b70-6564cd7dcf-j6jhv | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-04b70-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-04b70 |
InferenceGraphReady |
InferenceGraph [sequence-graph-04b70] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-04b70 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-04b70-6564cd7dcf from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-04b70-6564cd7dcf-j6jhv |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:0c87e1a125451a8a0f1f46fadc66ac5afc05f5cb73608aee9bffb1f827822f57" already present on machine | |
kserve-ci-e2e-test |
multus |
sequence-graph-04b70-6564cd7dcf-j6jhv |
AddedInterface |
Add eth0 [10.132.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-04b70-6564cd7dcf-j6jhv |
Created |
Created container: sequence-graph-04b70 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-04b70-6564cd7dcf-j6jhv |
Started |
Started container sequence-graph-04b70 | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-04b70-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-04b70-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-04b70 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-04b70 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-e0a3a-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-e0a3a-predictor-6f4c696847 from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e0a3a |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-e0a3a": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2aa9e-predictor-7ff7b8dffd-82f85 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2aa9e-predictor-7ff7b8dffd-82f85 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-2aa9e-744c775558-zn5kc |
Killing |
Stopping container ensemble-graph-2aa9e | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e0a3a |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-e0a3a": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-e0a3a": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-e0a3a-predictor-6f4c696847 |
SuccessfulCreate |
Created pod: success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-e0a3a-predictor-64d96fc94 |
SuccessfulCreate |
Created pod: error-404-isvc-e0a3a-predictor-64d96fc94-zpgcc | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-e0a3a-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-e0a3a-predictor-64d96fc94 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e0a3a-predictor-64d96fc94-zpgcc |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-e0a3a-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e0a3a |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-e0a3a": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e0a3a |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-e0a3a": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-e0a3a": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-e0a3a-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg |
AddedInterface |
Add eth0 [10.132.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-e0a3a-predictor-64d96fc94-zpgcc |
AddedInterface |
Add eth0 [10.133.0.40/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e0a3a-predictor-64d96fc94-zpgcc |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1919" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e0a3a-predictor-64d96fc94-zpgcc |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1919" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e0a3a-predictor-64d96fc94-zpgcc |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e0a3a-predictor-64d96fc94-zpgcc |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e0a3a-predictor-64d96fc94-zpgcc |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e0a3a-predictor-64d96fc94-zpgcc |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.26:8643/healthz": dial tcp 10.132.0.26:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.26:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-2aa9e-744c775558-zn5kc |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e0a3a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e0a3a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e0a3a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e0a3a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.31:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-e0a3a-predictor-64d96fc94-zpgcc |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.40:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e0a3a |
InferenceServiceReady |
InferenceService [error-404-isvc-e0a3a] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e0a3a |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e0a3a |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e0a3a |
InferenceServiceReady |
InferenceService [success-200-isvc-e0a3a] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-8365d-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-8365d-predictor-69558d5f59 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-04b70-predictor-697dcfdbb9-k58nf |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-04b70-predictor-5cff8969d4-8k7db |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-04b70-predictor-5cff8969d4-8k7db |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-04b70-6564cd7dcf-j6jhv |
Killing |
Stopping container sequence-graph-04b70 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-8365d-predictor-7567c8f6b |
SuccessfulCreate |
Created pod: error-404-isvc-8365d-predictor-7567c8f6b-jgvv5 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-8365d-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-8365d-predictor-7567c8f6b from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-8365d-predictor-69558d5f59 |
SuccessfulCreate |
Created pod: success-200-isvc-8365d-predictor-69558d5f59-868nq | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-04b70-predictor-697dcfdbb9-k58nf |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8365d-predictor-69558d5f59-868nq |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8365d-predictor-7567c8f6b-jgvv5 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8365d-predictor-69558d5f59-868nq |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8365d-predictor-69558d5f59-868nq |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8365d-predictor-69558d5f59-868nq |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-8365d-predictor-7567c8f6b-jgvv5 |
AddedInterface |
Add eth0 [10.133.0.41/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8365d-predictor-69558d5f59-868nq |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8365d-predictor-69558d5f59-868nq |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1919" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-8365d-predictor-69558d5f59-868nq |
AddedInterface |
Add eth0 [10.132.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8365d-predictor-7567c8f6b-jgvv5 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1919" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8365d-predictor-7567c8f6b-jgvv5 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8365d-predictor-7567c8f6b-jgvv5 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8365d-predictor-7567c8f6b-jgvv5 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8365d-predictor-7567c8f6b-jgvv5 |
Created |
Created container: kube-rbac-proxy | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e0a3a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-04b70-predictor-5cff8969d4-8k7db |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.39:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-04b70-predictor-697dcfdbb9-k58nf |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.28:8080: connect: connection refused |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-e0a3a |
ScalingReplicaSet |
Scaled up replica set splitter-graph-e0a3a-6b8c749bb4 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-04b70-predictor-5cff8969d4-8k7db |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.39:8643/healthz": dial tcp 10.133.0.39:8643: connect: connection refused | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-e0a3a-6b8c749bb4 |
SuccessfulCreate |
Created pod: splitter-graph-e0a3a-6b8c749bb4-pbx25 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e0a3a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-04b70-predictor-697dcfdbb9-k58nf |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.28:8643/healthz": dial tcp 10.132.0.28:8643: connect: connection refused | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e0a3a-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e0a3a-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-e0a3a |
InferenceGraphReady |
InferenceGraph [splitter-graph-e0a3a] is Ready | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-e0a3a-6b8c749bb4-pbx25 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "splitter-graph-e0a3a-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-e0a3a-6b8c749bb4-pbx25 |
Started |
Started container splitter-graph-e0a3a | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-e0a3a-6b8c749bb4-pbx25 |
Created |
Created container: splitter-graph-e0a3a | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-e0a3a-6b8c749bb4-pbx25 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:0c87e1a125451a8a0f1f46fadc66ac5afc05f5cb73608aee9bffb1f827822f57" already present on machine | |
kserve-ci-e2e-test |
multus |
splitter-graph-e0a3a-6b8c749bb4-pbx25 |
AddedInterface |
Add eth0 [10.132.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-e0a3a-6b8c749bb4-pbx25 |
Killing |
Stopping container splitter-graph-e0a3a | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-1a268-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-1a268-predictor-7c75dbcd8c |
SuccessfulCreate |
Created pod: success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-1a268-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-1a268-predictor-7c75dbcd8c from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1a268-predictor-58cf6c8694-2nfg5 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-1a268-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e0a3a-predictor-64d96fc94-zpgcc |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e0a3a-predictor-64d96fc94-zpgcc |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-1a268-predictor-58cf6c8694 |
SuccessfulCreate |
Created pod: error-404-isvc-1a268-predictor-58cf6c8694-2nfg5 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-1a268-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-1a268-predictor-58cf6c8694 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1919" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1a268-predictor-58cf6c8694-2nfg5 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1a268-predictor-58cf6c8694-2nfg5 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s |
AddedInterface |
Add eth0 [10.132.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1a268-predictor-58cf6c8694-2nfg5 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1a268-predictor-58cf6c8694-2nfg5 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1a268-predictor-58cf6c8694-2nfg5 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1919" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1a268-predictor-58cf6c8694-2nfg5 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-1a268-predictor-58cf6c8694-2nfg5 |
AddedInterface |
Add eth0 [10.133.0.42/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e0a3a-predictor-64d96fc94-zpgcc |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.40:8643/healthz": dial tcp 10.133.0.40:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-04b70-6564cd7dcf-j6jhv |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8365d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8365d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8365d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8365d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-8365d-predictor-7567c8f6b-jgvv5 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.41:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-8365d-predictor-69558d5f59-868nq |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.32:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-e0a3a-6b8c749bb4-pbx25 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1a268-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1a268-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1a268-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1a268-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-8365d |
InferenceServiceReady |
InferenceService [error-404-isvc-8365d] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-8365d |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-8365d |
InferenceServiceReady |
InferenceService [success-200-isvc-8365d] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-8365d |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-1a268-predictor-58cf6c8694-2nfg5 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.42:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.34:8080: connect: connection refused |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8365d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8365d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-8365d |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "switch-graph-8365d": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-8365d |
UpdateFailed |
Failed to update status for InferenceGraph "switch-graph-8365d": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "switch-graph-8365d": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-8365d |
InferenceGraphReady |
InferenceGraph [switch-graph-8365d] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-8365d |
ScalingReplicaSet |
Scaled up replica set switch-graph-8365d-79d674c554 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-8365d-79d674c554 |
SuccessfulCreate |
Created pod: switch-graph-8365d-79d674c554-6js2m | |
kserve-ci-e2e-test |
kubelet |
switch-graph-8365d-79d674c554-6js2m |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-8365d-serving-cert" not found | |
kserve-ci-e2e-test |
multus |
switch-graph-8365d-79d674c554-6js2m |
AddedInterface |
Add eth0 [10.132.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-8365d-79d674c554-6js2m |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:0c87e1a125451a8a0f1f46fadc66ac5afc05f5cb73608aee9bffb1f827822f57" already present on machine | |
kserve-ci-e2e-test |
kubelet |
switch-graph-8365d-79d674c554-6js2m |
Created |
Created container: switch-graph-8365d | |
kserve-ci-e2e-test |
kubelet |
switch-graph-8365d-79d674c554-6js2m |
Started |
Started container switch-graph-8365d | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-1a268 |
InferenceServiceReady |
InferenceService [success-200-isvc-1a268] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-1a268 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-1a268 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-1a268 |
InferenceServiceReady |
InferenceService [error-404-isvc-1a268] is Ready | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-1a268-7579f4dc47-xgz4s |
Created |
Created container: splitter-graph-1a268 | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-1a268-7579f4dc47-xgz4s |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:0c87e1a125451a8a0f1f46fadc66ac5afc05f5cb73608aee9bffb1f827822f57" already present on machine | |
kserve-ci-e2e-test |
multus |
splitter-graph-1a268-7579f4dc47-xgz4s |
AddedInterface |
Add eth0 [10.132.0.36/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-1a268-7579f4dc47-xgz4s |
Started |
Started container splitter-graph-1a268 | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-1a268 |
ScalingReplicaSet |
Scaled up replica set splitter-graph-1a268-7579f4dc47 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-1a268-7579f4dc47 |
SuccessfulCreate |
Created pod: splitter-graph-1a268-7579f4dc47-xgz4s | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-1a268 |
InferenceGraphReady |
InferenceGraph [splitter-graph-1a268] is Ready | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8365d-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8365d-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1a268-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1a268-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1a268-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1a268-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-1a268 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-1a268 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-8365d |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-8365d |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1a268-predictor-58cf6c8694-2nfg5 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1a268-predictor-58cf6c8694-2nfg5 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-1a268-7579f4dc47-xgz4s |
Killing |
Stopping container splitter-graph-1a268 | |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-1a268-7579f4dc47-xgz4s |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8365d-predictor-7567c8f6b-jgvv5 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
switch-graph-8365d-79d674c554-6js2m |
Killing |
Stopping container switch-graph-8365d | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8365d-predictor-7567c8f6b-jgvv5 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8365d-predictor-69558d5f59-868nq |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8365d-predictor-69558d5f59-868nq |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8365d-predictor-7567c8f6b-jgvv5 |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.41:8643/healthz": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-8365d-79d674c554-6js2m |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |