| Time | Namespace | Component | RelatedObject | Reason | Message |
|---|---|---|---|---|---|
kserve-ci-e2e-test |
switch-graph-15f71-6d86bc76cd-7x6g6 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6 to ip-10-0-133-99.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-d2d49-predictor-c9cf5cd84-q4dlk |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-d2d49-predictor-c9cf5cd84-q4dlk to ip-10-0-140-67.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-1593f-predictor-79647f4cf4-5mc46 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-1593f-predictor-79647f4cf4-5mc46 to ip-10-0-140-67.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-0e2c5-5df677ff7b-cvghx |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx to ip-10-0-133-99.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-15f71-predictor-784478884b-fqkdn |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-15f71-predictor-784478884b-fqkdn to ip-10-0-140-67.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-21301-predictor-68796d9f44-xxrjr |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-21301-predictor-68796d9f44-xxrjr to ip-10-0-140-67.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-d2d49-6548fd667c-bsphk |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk to ip-10-0-133-99.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-1-predictor-57779f596f-6hgbk |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-57779f596f-6hgbk to ip-10-0-140-67.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-e3b29-879b5696f-mrtph |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph to ip-10-0-133-99.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-617e5-69d5f987b8-qz8dk |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk to ip-10-0-133-99.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-e4920-59fb4569b6-fl2qz |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz to ip-10-0-133-99.ec2.internal | ||
kserve-ci-e2e-test |
isvc-xgboost-graph-predictor-555fc69d56-jvn9k |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-jvn9k to ip-10-0-140-67.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-e3b29-predictor-596b8c9d77-b79mj |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-e3b29-predictor-596b8c9d77-b79mj to ip-10-0-128-29.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-34033-predictor-6dcb7cd944-jm6v8 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-34033-predictor-6dcb7cd944-jm6v8 to ip-10-0-140-67.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-e4920-predictor-75647c67f-x4zcq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-e4920-predictor-75647c67f-x4zcq to ip-10-0-128-29.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-34033-predictor-b4f8cfdcc-p2gj8 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-34033-predictor-b4f8cfdcc-p2gj8 to ip-10-0-128-29.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-15f71-predictor-5c75fd5cf5-mz2q8 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-15f71-predictor-5c75fd5cf5-mz2q8 to ip-10-0-128-29.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-2-predictor-6b4888bd86-vt525 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6b4888bd86-vt525 to ip-10-0-140-67.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-1593f-5c79fd86d5-9xg84 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84 to ip-10-0-133-99.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-35a06-predictor-7d699577b8-8szfb |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-35a06-predictor-7d699577b8-8szfb to ip-10-0-140-67.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-21301-predictor-6f496df769-xkcv9 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-21301-predictor-6f496df769-xkcv9 to ip-10-0-128-29.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-35a06-predictor-59f949b8b5-wkdgn |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-35a06-predictor-59f949b8b5-wkdgn to ip-10-0-128-29.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-1593f-predictor-599ddd5c95-jfd8z |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-1593f-predictor-599ddd5c95-jfd8z to ip-10-0-128-29.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-0e2c5-predictor-7cb46d9cfc-7f2gl |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-0e2c5-predictor-7cb46d9cfc-7f2gl to ip-10-0-128-29.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-35a06-7fd9cccc6d-k2gnt |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt to ip-10-0-133-99.ec2.internal | ||
kserve-ci-e2e-test |
model-chainer-7c9bd794bb-kw2xs |
Scheduled |
Successfully assigned kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs to ip-10-0-133-99.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-21301-7fb87c6d96-cn98k |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k to ip-10-0-133-99.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-0e2c5-predictor-7956b5b96-jwqzd |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-0e2c5-predictor-7956b5b96-jwqzd to ip-10-0-140-67.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-d2d49-predictor-9686f48b9-cnvdw |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-d2d49-predictor-9686f48b9-cnvdw to ip-10-0-128-29.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-617e5-predictor-6dd67cff67-zcf86 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-617e5-predictor-6dd67cff67-zcf86 to ip-10-0-128-29.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-34033-5489c4744b-q5rmp |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp to ip-10-0-133-99.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-e4920-predictor-868cb5f55d-t4824 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-e4920-predictor-868cb5f55d-t4824 to ip-10-0-140-67.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-e3b29-predictor-7b4b4f756d-mzxqg |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-e3b29-predictor-7b4b4f756d-mzxqg to ip-10-0-140-67.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-617e5-predictor-776d98c8-5r9t7 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-617e5-predictor-776d98c8-5r9t7 to ip-10-0-140-67.ec2.internal | ||
kserve-ci-e2e-test |
multus |
success-200-isvc-35a06-predictor-7d699577b8-8szfb |
AddedInterface |
Add eth0 [10.134.0.25/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-1-predictor-57779f596f |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-1-predictor-57779f596f-6hgbk | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-35a06-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-35a06-predictor-59f949b8b5 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-35a06-predictor-7d699577b8 |
SuccessfulCreate |
Created pod: success-200-isvc-35a06-predictor-7d699577b8-8szfb | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-2-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-2-predictor-6b4888bd86 from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-1-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-1-predictor-57779f596f from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-35a06-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-35a06-predictor-7d699577b8 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-35a06-predictor-7d699577b8-8szfb |
Pulling |
Pulling image "quay.io/opendatahub/success-200-isvc:odh-pr-1987" | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-xgboost-graph-predictor-555fc69d56 |
SuccessfulCreate |
Created pod: isvc-xgboost-graph-predictor-555fc69d56-jvn9k | |
kserve-ci-e2e-test |
deployment-controller |
isvc-xgboost-graph-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-xgboost-graph-predictor-555fc69d56 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-35a06-predictor-59f949b8b5 |
SuccessfulCreate |
Created pod: error-404-isvc-35a06-predictor-59f949b8b5-wkdgn | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-57779f596f-6hgbk |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6b4888bd86-vt525 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "isvc-sklearn-graph-2-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-1-predictor-57779f596f-6hgbk |
AddedInterface |
Add eth0 [10.134.0.26/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-57779f596f-6hgbk |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:d22994b609214bce65bb4b930496142821ad1e113aba3930763bfdc84da5f0ea" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-jvn9k |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "isvc-xgboost-graph-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-2-predictor-6b4888bd86 |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-2-predictor-6b4888bd86-vt525 | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-57779f596f-6hgbk |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-35a06-predictor-59f949b8b5-wkdgn |
Pulling |
Pulling image "quay.io/opendatahub/error-404-isvc:odh-pr-1987" | |
kserve-ci-e2e-test |
multus |
error-404-isvc-35a06-predictor-59f949b8b5-wkdgn |
AddedInterface |
Add eth0 [10.133.0.16/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-jvn9k |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6b4888bd86-vt525 |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
multus |
isvc-xgboost-graph-predictor-555fc69d56-jvn9k |
AddedInterface |
Add eth0 [10.134.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-jvn9k |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:d22994b609214bce65bb4b930496142821ad1e113aba3930763bfdc84da5f0ea" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6b4888bd86-vt525 |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6b4888bd86-vt525 |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:d22994b609214bce65bb4b930496142821ad1e113aba3930763bfdc84da5f0ea" already present on machine | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-2-predictor-6b4888bd86-vt525 |
AddedInterface |
Add eth0 [10.134.0.28/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-jvn9k |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-57779f596f-6hgbk |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1987" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-jvn9k |
Pulling |
Pulling image "kserve/xgbserver:latest" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6b4888bd86-vt525 |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1987" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-35a06-predictor-59f949b8b5-wkdgn |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-35a06-predictor-59f949b8b5-wkdgn |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-35a06-predictor-59f949b8b5-wkdgn |
Pulled |
Successfully pulled image "quay.io/opendatahub/error-404-isvc:odh-pr-1987" in 11.508s (11.508s including waiting). Image size: 1357341778 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-35a06-predictor-59f949b8b5-wkdgn |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-35a06-predictor-7d699577b8-8szfb |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-35a06-predictor-7d699577b8-8szfb |
Pulled |
Successfully pulled image "quay.io/opendatahub/success-200-isvc:odh-pr-1987" in 12.682s (12.682s including waiting). Image size: 1358345301 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-35a06-predictor-7d699577b8-8szfb |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-35a06-predictor-7d699577b8-8szfb |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-35a06-predictor-59f949b8b5-wkdgn |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.836s (2.836s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-35a06-predictor-59f949b8b5-wkdgn |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-35a06-predictor-59f949b8b5-wkdgn |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-57779f596f-6hgbk |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1987" in 16.475s (16.475s including waiting). Image size: 1580716956 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6b4888bd86-vt525 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6b4888bd86-vt525 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6b4888bd86-vt525 |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1987" in 12.812s (12.812s including waiting). Image size: 1580716956 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6b4888bd86-vt525 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6b4888bd86-vt525 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-35a06-predictor-7d699577b8-8szfb |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-35a06-predictor-7d699577b8-8szfb |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-35a06-predictor-7d699577b8-8szfb |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 9.426s (9.426s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-57779f596f-6hgbk |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6b4888bd86-vt525 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-57779f596f-6hgbk |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-57779f596f-6hgbk |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-57779f596f-6hgbk |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-57779f596f-6hgbk |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-jvn9k |
Pulled |
Successfully pulled image "kserve/xgbserver:latest" in 22.713s (22.713s including waiting). Image size: 1332365570 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-jvn9k |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-jvn9k |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-jvn9k |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-jvn9k |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-jvn9k |
Created |
Created container: kserve-container | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-35a06-predictor-7d699577b8-8szfb |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.25:8080: connect: connection refused |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-35a06-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-35a06-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-35a06-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-35a06-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-35a06 |
InferenceServiceReady |
InferenceService [error-404-isvc-35a06] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-35a06 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-35a06 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-35a06 |
InferenceServiceReady |
InferenceService [success-200-isvc-35a06] is Ready | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-35a06-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-35a06-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
kubelet |
switch-graph-35a06-7fd9cccc6d-k2gnt |
Pulling |
Pulling image "quay.io/opendatahub/kserve-router@sha256:8bcf19d9ed90070ea776e0c3189bb53a4b1362b2d66bf974f83dca260544e4db" | |
kserve-ci-e2e-test |
multus |
switch-graph-35a06-7fd9cccc6d-k2gnt |
AddedInterface |
Add eth0 [10.132.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-35a06 |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "switch-graph-35a06": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-35a06 |
UpdateFailed |
Failed to update status for InferenceGraph "switch-graph-35a06": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "switch-graph-35a06": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-35a06 |
InferenceGraphReady |
InferenceGraph [switch-graph-35a06] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-35a06 |
ScalingReplicaSet |
Scaled up replica set switch-graph-35a06-7fd9cccc6d from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-35a06-7fd9cccc6d |
SuccessfulCreate |
Created pod: switch-graph-35a06-7fd9cccc6d-k2gnt | |
kserve-ci-e2e-test |
kubelet |
switch-graph-35a06-7fd9cccc6d-k2gnt |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-router@sha256:8bcf19d9ed90070ea776e0c3189bb53a4b1362b2d66bf974f83dca260544e4db" in 1.938s (1.938s including waiting). Image size: 211458663 bytes. | |
kserve-ci-e2e-test |
kubelet |
switch-graph-35a06-7fd9cccc6d-k2gnt |
Created |
Created container: switch-graph-35a06 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-35a06-7fd9cccc6d-k2gnt |
Started |
Started container switch-graph-35a06 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-15f71-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-15f71-predictor-5c75fd5cf5 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-15f71-predictor-5c75fd5cf5 |
SuccessfulCreate |
Created pod: error-404-isvc-15f71-predictor-5c75fd5cf5-mz2q8 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-35a06-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-35a06-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
switch-graph-35a06-7fd9cccc6d-k2gnt |
Killing |
Stopping container switch-graph-35a06 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-35a06-predictor-59f949b8b5-wkdgn |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-35a06-predictor-59f949b8b5-wkdgn |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-35a06-predictor-7d699577b8-8szfb |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-35a06-predictor-7d699577b8-8szfb |
Killing |
Stopping container kserve-container | |
| (x8) | kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-jvn9k |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.27:8080: connect: connection refused |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-15f71-predictor-784478884b |
SuccessfulCreate |
Created pod: success-200-isvc-15f71-predictor-784478884b-fqkdn | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-15f71-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-15f71-predictor-784478884b from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-15f71-predictor-784478884b-fqkdn |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-15f71-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-15f71-predictor-5c75fd5cf5-mz2q8 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1987" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-15f71-predictor-5c75fd5cf5-mz2q8 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-15f71-predictor-784478884b-fqkdn |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-15f71-predictor-784478884b-fqkdn |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
success-200-isvc-15f71-predictor-784478884b-fqkdn |
AddedInterface |
Add eth0 [10.134.0.29/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-15f71-predictor-784478884b-fqkdn |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1987" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-15f71-predictor-5c75fd5cf5-mz2q8 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-15f71-predictor-784478884b-fqkdn |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-15f71-predictor-5c75fd5cf5-mz2q8 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-15f71-predictor-5c75fd5cf5-mz2q8 |
AddedInterface |
Add eth0 [10.133.0.17/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-15f71-predictor-5c75fd5cf5-mz2q8 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-15f71-predictor-5c75fd5cf5-mz2q8 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-15f71-predictor-784478884b-fqkdn |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-15f71-predictor-784478884b-fqkdn |
Created |
Created container: kube-rbac-proxy | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-35a06-predictor-59f949b8b5-wkdgn |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.16:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-35a06-predictor-59f949b8b5-wkdgn |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.16:8643/healthz": dial tcp 10.133.0.16:8643: connect: connection refused | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
InferenceServiceReady |
InferenceService [isvc-xgboost-graph] is Ready | |
| (x10) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6b4888bd86-vt525 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.28:8080: connect: connection refused |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-1] is Ready | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-2] is Ready | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-35a06-7fd9cccc6d-k2gnt |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-15f71-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-15f71-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
model-chainer-7c9bd794bb-kw2xs |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:8bcf19d9ed90070ea776e0c3189bb53a4b1362b2d66bf974f83dca260544e4db" already present on machine | |
kserve-ci-e2e-test |
replicaset-controller |
model-chainer-7c9bd794bb |
SuccessfulCreate |
Created pod: model-chainer-7c9bd794bb-kw2xs | |
kserve-ci-e2e-test |
kubelet |
model-chainer-7c9bd794bb-kw2xs |
Started |
Started container model-chainer | |
kserve-ci-e2e-test |
multus |
model-chainer-7c9bd794bb-kw2xs |
AddedInterface |
Add eth0 [10.132.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
model-chainer-7c9bd794bb-kw2xs |
Created |
Created container: model-chainer | |
kserve-ci-e2e-test |
deployment-controller |
model-chainer |
ScalingReplicaSet |
Scaled up replica set model-chainer-7c9bd794bb from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
model-chainer |
InferenceGraphReady |
InferenceGraph [model-chainer] is Ready | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-15f71-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-15f71-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-15f71 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-15f71 |
InferenceServiceReady |
InferenceService [error-404-isvc-15f71] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-15f71 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-15f71 |
InferenceServiceReady |
InferenceService [success-200-isvc-15f71] is Ready | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-57779f596f-6hgbk |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-57779f596f-6hgbk |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
model-chainer-7c9bd794bb-kw2xs |
Killing |
Stopping container model-chainer | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6b4888bd86-vt525 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6b4888bd86-vt525 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-21301-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-21301-predictor-68796d9f44 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-21301-predictor-68796d9f44 |
SuccessfulCreate |
Created pod: success-200-isvc-21301-predictor-68796d9f44-xxrjr | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-21301-predictor-6f496df769 |
SuccessfulCreate |
Created pod: error-404-isvc-21301-predictor-6f496df769-xkcv9 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-21301-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-21301-predictor-6f496df769 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-jvn9k |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-jvn9k |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-21301-predictor-6f496df769-xkcv9 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
| (x10) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-57779f596f-6hgbk |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.26:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-21301-predictor-68796d9f44-xxrjr |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-21301-predictor-68796d9f44-xxrjr |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-21301-predictor-68796d9f44-xxrjr |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-21301-predictor-68796d9f44-xxrjr |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1987" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-21301-predictor-68796d9f44-xxrjr |
AddedInterface |
Add eth0 [10.134.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-21301-predictor-68796d9f44-xxrjr |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-21301-predictor-6f496df769-xkcv9 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-21301-predictor-6f496df769-xkcv9 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-21301-predictor-6f496df769-xkcv9 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-57779f596f-6hgbk |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.26:8643/healthz": dial tcp 10.134.0.26:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-21301-predictor-6f496df769-xkcv9 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-21301-predictor-6f496df769-xkcv9 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1987" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-21301-predictor-6f496df769-xkcv9 |
AddedInterface |
Add eth0 [10.133.0.18/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-21301-predictor-68796d9f44-xxrjr |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-6b4888bd86-vt525 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.28:8643/healthz": dial tcp 10.134.0.28:8643: connect: connection refused | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-15f71-6d86bc76cd |
SuccessfulCreate |
Created pod: switch-graph-15f71-6d86bc76cd-7x6g6 | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-15f71 |
ScalingReplicaSet |
Scaled up replica set switch-graph-15f71-6d86bc76cd from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-15f71 |
InferenceGraphReady |
InferenceGraph [switch-graph-15f71] is Ready | |
kserve-ci-e2e-test |
kubelet |
switch-graph-15f71-6d86bc76cd-7x6g6 |
Created |
Created container: switch-graph-15f71 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-15f71-6d86bc76cd-7x6g6 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:8bcf19d9ed90070ea776e0c3189bb53a4b1362b2d66bf974f83dca260544e4db" already present on machine | |
kserve-ci-e2e-test |
kubelet |
switch-graph-15f71-6d86bc76cd-7x6g6 |
Started |
Started container switch-graph-15f71 | |
kserve-ci-e2e-test |
multus |
switch-graph-15f71-6d86bc76cd-7x6g6 |
AddedInterface |
Add eth0 [10.132.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-15f71 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-15f71 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
model-chainer-7c9bd794bb-kw2xs |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-21301-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-21301-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-21301-predictor-6f496df769-xkcv9 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.18:8080: connect: connection refused |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-15f71 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-15f71-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-15f71-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-15f71-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-15f71-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-15f71 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-21301 |
InferenceServiceReady |
InferenceService [success-200-isvc-21301] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-21301 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-21301 |
InferenceServiceReady |
InferenceService [error-404-isvc-21301] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-21301 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-21301 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-21301-7fb87c6d96 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-21301 |
InferenceGraphReady |
InferenceGraph [sequence-graph-21301] is Ready | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-21301-7fb87c6d96-cn98k |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "sequence-graph-21301-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-21301-7fb87c6d96 |
SuccessfulCreate |
Created pod: sequence-graph-21301-7fb87c6d96-cn98k | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-21301-7fb87c6d96-cn98k |
Created |
Created container: sequence-graph-21301 | |
kserve-ci-e2e-test |
multus |
sequence-graph-21301-7fb87c6d96-cn98k |
AddedInterface |
Add eth0 [10.132.0.36/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-21301-7fb87c6d96-cn98k |
Started |
Started container sequence-graph-21301 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-21301-7fb87c6d96-cn98k |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:8bcf19d9ed90070ea776e0c3189bb53a4b1362b2d66bf974f83dca260544e4db" already present on machine | |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-21301-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-21301-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-21301 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-21301-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-21301 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-21301-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-15f71-predictor-784478884b-fqkdn |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
switch-graph-15f71-6d86bc76cd-7x6g6 |
Killing |
Stopping container switch-graph-15f71 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-15f71-predictor-784478884b-fqkdn |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-1593f-predictor-79647f4cf4 |
SuccessfulCreate |
Created pod: success-200-isvc-1593f-predictor-79647f4cf4-5mc46 | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-1593f-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-1593f-predictor-79647f4cf4 from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-1593f-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-1593f-predictor-599ddd5c95 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-15f71-predictor-5c75fd5cf5-mz2q8 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-15f71-predictor-5c75fd5cf5-mz2q8 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-1593f-predictor-599ddd5c95 |
SuccessfulCreate |
Created pod: error-404-isvc-1593f-predictor-599ddd5c95-jfd8z | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1593f-predictor-599ddd5c95-jfd8z |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-1593f-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1593f-predictor-79647f4cf4-5mc46 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1593f-predictor-79647f4cf4-5mc46 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1593f-predictor-79647f4cf4-5mc46 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1593f-predictor-79647f4cf4-5mc46 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1593f-predictor-79647f4cf4-5mc46 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1593f-predictor-79647f4cf4-5mc46 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1987" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-1593f-predictor-79647f4cf4-5mc46 |
AddedInterface |
Add eth0 [10.134.0.31/23] from ovn-kubernetes | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-15f71-predictor-5c75fd5cf5-mz2q8 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.17:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1593f-predictor-599ddd5c95-jfd8z |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1593f-predictor-599ddd5c95-jfd8z |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1593f-predictor-599ddd5c95-jfd8z |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1593f-predictor-599ddd5c95-jfd8z |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-15f71-predictor-5c75fd5cf5-mz2q8 |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.17:8643/healthz": dial tcp 10.133.0.17:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1593f-predictor-599ddd5c95-jfd8z |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1593f-predictor-599ddd5c95-jfd8z |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1987" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-1593f-predictor-599ddd5c95-jfd8z |
AddedInterface |
Add eth0 [10.133.0.19/23] from ovn-kubernetes | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-15f71-predictor-784478884b-fqkdn |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.29:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-15f71-predictor-784478884b-fqkdn |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.29:8643/healthz": dial tcp 10.134.0.29:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-15f71-6d86bc76cd-7x6g6 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1593f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1593f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-1593f-predictor-599ddd5c95-jfd8z |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.19:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-1593f-predictor-79647f4cf4-5mc46 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.31:8080: connect: connection refused |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1593f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1593f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-1593f |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-1593f |
InferenceServiceReady |
InferenceService [error-404-isvc-1593f] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-1593f |
InferenceServiceReady |
InferenceService [success-200-isvc-1593f] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-1593f |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-34033-predictor-6dcb7cd944 |
SuccessfulCreate |
Created pod: success-200-isvc-34033-predictor-6dcb7cd944-jm6v8 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-34033-predictor-b4f8cfdcc |
SuccessfulCreate |
Created pod: error-404-isvc-34033-predictor-b4f8cfdcc-p2gj8 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-21301-predictor-6f496df769-xkcv9 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-21301-predictor-68796d9f44-xxrjr |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-34033-predictor-b4f8cfdcc-p2gj8 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-34033-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-21301-predictor-68796d9f44-xxrjr |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-34033-predictor-6dcb7cd944-jm6v8 |
AddedInterface |
Add eth0 [10.134.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-34033-predictor-6dcb7cd944-jm6v8 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1987" already present on machine | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-21301-7fb87c6d96-cn98k |
Killing |
Stopping container sequence-graph-21301 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-21301-predictor-6f496df769-xkcv9 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-34033-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-34033-predictor-6dcb7cd944 from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-34033-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-34033-predictor-b4f8cfdcc from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-34033-predictor-b4f8cfdcc-p2gj8 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-34033-predictor-b4f8cfdcc-p2gj8 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-34033-predictor-6dcb7cd944-jm6v8 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-34033-predictor-b4f8cfdcc-p2gj8 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-34033-predictor-b4f8cfdcc-p2gj8 |
AddedInterface |
Add eth0 [10.133.0.20/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-34033-predictor-b4f8cfdcc-p2gj8 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1987" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-34033-predictor-b4f8cfdcc-p2gj8 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-34033-predictor-6dcb7cd944-jm6v8 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-34033-predictor-6dcb7cd944-jm6v8 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-34033-predictor-b4f8cfdcc-p2gj8 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-34033-predictor-6dcb7cd944-jm6v8 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-34033-predictor-6dcb7cd944-jm6v8 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-21301-predictor-68796d9f44-xxrjr |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.30:8643/healthz": dial tcp 10.134.0.30:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-21301-predictor-68796d9f44-xxrjr |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.30:8080: connect: connection refused |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-1593f |
InferenceGraphReady |
InferenceGraph [ensemble-graph-1593f] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-1593f |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-1593f-5c79fd86d5 from 0 to 1 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1593f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-1593f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-1593f-5c79fd86d5 |
SuccessfulCreate |
Created pod: ensemble-graph-1593f-5c79fd86d5-9xg84 | |
kserve-ci-e2e-test |
multus |
ensemble-graph-1593f-5c79fd86d5-9xg84 |
AddedInterface |
Add eth0 [10.132.0.37/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-1593f-5c79fd86d5-9xg84 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:8bcf19d9ed90070ea776e0c3189bb53a4b1362b2d66bf974f83dca260544e4db" already present on machine | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-1593f-5c79fd86d5-9xg84 |
Created |
Created container: ensemble-graph-1593f | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1593f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-1593f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-1593f-5c79fd86d5-9xg84 |
Started |
Started container ensemble-graph-1593f | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-1593f-5c79fd86d5-9xg84 |
Killing |
Stopping container ensemble-graph-1593f | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-e4920-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-e4920-predictor-75647c67f from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1593f-predictor-79647f4cf4-5mc46 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-e4920-predictor-868cb5f55d |
SuccessfulCreate |
Created pod: success-200-isvc-e4920-predictor-868cb5f55d-t4824 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-e4920-predictor-75647c67f |
SuccessfulCreate |
Created pod: error-404-isvc-e4920-predictor-75647c67f-x4zcq | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1593f-predictor-599ddd5c95-jfd8z |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e4920-predictor-868cb5f55d-t4824 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-e4920-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1593f-predictor-79647f4cf4-5mc46 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-e4920-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-e4920-predictor-868cb5f55d from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1593f-predictor-599ddd5c95-jfd8z |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e4920-predictor-868cb5f55d-t4824 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e4920-predictor-868cb5f55d-t4824 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e4920-predictor-868cb5f55d-t4824 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
success-200-isvc-e4920-predictor-868cb5f55d-t4824 |
AddedInterface |
Add eth0 [10.134.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e4920-predictor-868cb5f55d-t4824 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e4920-predictor-868cb5f55d-t4824 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-e4920-predictor-75647c67f-x4zcq |
AddedInterface |
Add eth0 [10.133.0.21/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e4920-predictor-75647c67f-x4zcq |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e4920-predictor-868cb5f55d-t4824 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1987" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e4920-predictor-75647c67f-x4zcq |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1987" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e4920-predictor-75647c67f-x4zcq |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e4920-predictor-75647c67f-x4zcq |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e4920-predictor-75647c67f-x4zcq |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e4920-predictor-75647c67f-x4zcq |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-1593f-predictor-79647f4cf4-5mc46 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.31:8643/healthz": dial tcp 10.134.0.31:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-1593f-predictor-599ddd5c95-jfd8z |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.19:8643/healthz": dial tcp 10.133.0.19:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-21301-7fb87c6d96-cn98k |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-34033-predictor-6dcb7cd944-jm6v8 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.32:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-34033-predictor-b4f8cfdcc-p2gj8 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.20:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-34033-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-34033-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-1593f-5c79fd86d5-9xg84 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-34033 |
InferenceServiceReady |
InferenceService [success-200-isvc-34033] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-34033 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-34033 |
InferenceServiceReady |
InferenceService [error-404-isvc-34033] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-34033 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e4920-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e4920-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e4920-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e4920-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-34033 |
InferenceGraphReady |
InferenceGraph [sequence-graph-34033] is Ready | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-34033-5489c4744b-q5rmp |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:8bcf19d9ed90070ea776e0c3189bb53a4b1362b2d66bf974f83dca260544e4db" already present on machine | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-34033-5489c4744b |
SuccessfulCreate |
Created pod: sequence-graph-34033-5489c4744b-q5rmp | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-34033-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-34033-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-34033-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-34033-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-34033 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-34033-5489c4744b from 0 to 1 | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-e4920-predictor-868cb5f55d-t4824 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.33:8080: connect: connection refused |
kserve-ci-e2e-test |
multus |
sequence-graph-34033-5489c4744b-q5rmp |
AddedInterface |
Add eth0 [10.132.0.38/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-34033-5489c4744b-q5rmp |
Created |
Created container: sequence-graph-34033 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-34033-5489c4744b-q5rmp |
Started |
Started container sequence-graph-34033 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e4920 |
InferenceServiceReady |
InferenceService [error-404-isvc-e4920] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e4920 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-34033-predictor-b4f8cfdcc-p2gj8 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-34033-predictor-b4f8cfdcc-p2gj8 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-34033-5489c4744b-q5rmp |
Killing |
Stopping container sequence-graph-34033 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e4920 |
InferenceServiceReady |
InferenceService [success-200-isvc-e4920] is Ready | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-34033-predictor-6dcb7cd944-jm6v8 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-e3b29-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-e3b29-predictor-7b4b4f756d from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-e3b29-predictor-7b4b4f756d |
SuccessfulCreate |
Created pod: success-200-isvc-e3b29-predictor-7b4b4f756d-mzxqg | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-34033-predictor-6dcb7cd944-jm6v8 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e3b29-predictor-596b8c9d77-b79mj |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e3b29-predictor-596b8c9d77-b79mj |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e3b29-predictor-7b4b4f756d-mzxqg |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e3b29-predictor-7b4b4f756d-mzxqg |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e3b29-predictor-7b4b4f756d-mzxqg |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1987" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-e3b29-predictor-7b4b4f756d-mzxqg |
AddedInterface |
Add eth0 [10.134.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e3b29-predictor-7b4b4f756d-mzxqg |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-e3b29-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-e3b29-predictor-596b8c9d77 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-e3b29-predictor-596b8c9d77 |
SuccessfulCreate |
Created pod: error-404-isvc-e3b29-predictor-596b8c9d77-b79mj | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e3b29-predictor-596b8c9d77-b79mj |
Started |
Started container kube-rbac-proxy | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e4920 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e3b29-predictor-7b4b4f756d-mzxqg |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e3b29-predictor-7b4b4f756d-mzxqg |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e3b29-predictor-596b8c9d77-b79mj |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-e3b29-predictor-596b8c9d77-b79mj |
AddedInterface |
Add eth0 [10.133.0.22/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e3b29-predictor-596b8c9d77-b79mj |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e3b29-predictor-596b8c9d77-b79mj |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1987" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-34033-predictor-6dcb7cd944-jm6v8 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.32:8643/healthz": dial tcp 10.134.0.32:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-e4920-59fb4569b6-fl2qz |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "ensemble-graph-e4920-serving-cert" not found | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-e4920 |
InferenceGraphReady |
InferenceGraph [ensemble-graph-e4920] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-e4920-59fb4569b6 |
SuccessfulCreate |
Created pod: ensemble-graph-e4920-59fb4569b6-fl2qz | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-e4920 |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-e4920-59fb4569b6 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-e4920-59fb4569b6-fl2qz |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:8bcf19d9ed90070ea776e0c3189bb53a4b1362b2d66bf974f83dca260544e4db" already present on machine | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-e4920-59fb4569b6-fl2qz |
Started |
Started container ensemble-graph-e4920 | |
kserve-ci-e2e-test |
multus |
ensemble-graph-e4920-59fb4569b6-fl2qz |
AddedInterface |
Add eth0 [10.132.0.39/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-e4920-59fb4569b6-fl2qz |
Created |
Created container: ensemble-graph-e4920 | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e3b29-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e3b29-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e4920-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e4920-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e4920-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e4920-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-34033-5489c4744b-q5rmp |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-e4920 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-e4920 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e3b29-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e3b29-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e3b29 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e3b29 |
InferenceServiceReady |
InferenceService [success-200-isvc-e3b29] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e3b29 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e3b29 |
InferenceServiceReady |
InferenceService [error-404-isvc-e3b29] is Ready | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-e3b29 |
InferenceGraphReady |
InferenceGraph [sequence-graph-e3b29] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-e3b29 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-e3b29-879b5696f from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-e3b29-879b5696f |
SuccessfulCreate |
Created pod: sequence-graph-e3b29-879b5696f-mrtph | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-e3b29-879b5696f-mrtph |
Started |
Started container sequence-graph-e3b29 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-e3b29-879b5696f-mrtph |
Created |
Created container: sequence-graph-e3b29 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-e3b29-879b5696f-mrtph |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:8bcf19d9ed90070ea776e0c3189bb53a4b1362b2d66bf974f83dca260544e4db" already present on machine | |
kserve-ci-e2e-test |
multus |
sequence-graph-e3b29-879b5696f-mrtph |
AddedInterface |
Add eth0 [10.132.0.40/23] from ovn-kubernetes | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e3b29-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-e3b29 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e3b29-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-e3b29 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e3b29-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e3b29-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-e3b29 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-e3b29 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-e4920-59fb4569b6-fl2qz |
Killing |
Stopping container ensemble-graph-e4920 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-617e5-predictor-6dd67cff67 |
SuccessfulCreate |
Created pod: error-404-isvc-617e5-predictor-6dd67cff67-zcf86 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e4920-predictor-868cb5f55d-t4824 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-617e5 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-617e5": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-617e5 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-617e5": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-617e5": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e4920-predictor-75647c67f-x4zcq |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-617e5-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-617e5-predictor-776d98c8 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-617e5-predictor-776d98c8 |
SuccessfulCreate |
Created pod: success-200-isvc-617e5-predictor-776d98c8-5r9t7 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-617e5-predictor-776d98c8-5r9t7 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1987" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-617e5-predictor-776d98c8-5r9t7 |
AddedInterface |
Add eth0 [10.134.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e4920-predictor-75647c67f-x4zcq |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e4920-predictor-868cb5f55d-t4824 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-617e5-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-617e5-predictor-6dd67cff67 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-617e5-predictor-6dd67cff67-zcf86 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-617e5-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-617e5-predictor-6dd67cff67-zcf86 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-617e5-predictor-6dd67cff67-zcf86 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-617e5-predictor-6dd67cff67-zcf86 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1987" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-617e5-predictor-6dd67cff67-zcf86 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-617e5-predictor-6dd67cff67-zcf86 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-617e5-predictor-6dd67cff67-zcf86 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-617e5-predictor-776d98c8-5r9t7 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-617e5-predictor-6dd67cff67-zcf86 |
AddedInterface |
Add eth0 [10.133.0.23/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-617e5-predictor-776d98c8-5r9t7 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-617e5-predictor-776d98c8-5r9t7 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-617e5-predictor-776d98c8-5r9t7 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-617e5-predictor-776d98c8-5r9t7 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e4920-predictor-75647c67f-x4zcq |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.21:8643/healthz": dial tcp 10.133.0.21:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-e4920-predictor-75647c67f-x4zcq |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.21:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-e4920-59fb4569b6-fl2qz |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-617e5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-617e5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-617e5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-617e5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-617e5-predictor-6dd67cff67-zcf86 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.23:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-617e5-predictor-776d98c8-5r9t7 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.35:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-617e5 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-617e5 |
InferenceServiceReady |
InferenceService [error-404-isvc-617e5] is Ready | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-e3b29-879b5696f-mrtph |
Killing |
Stopping container sequence-graph-e3b29 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d2d49 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-d2d49": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-d2d49-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-d2d49-predictor-9686f48b9 from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d2d49 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-d2d49": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-d2d49": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-617e5 |
InferenceServiceReady |
InferenceService [success-200-isvc-617e5] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-d2d49-predictor-9686f48b9 |
SuccessfulCreate |
Created pod: error-404-isvc-d2d49-predictor-9686f48b9-cnvdw | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e3b29-predictor-596b8c9d77-b79mj |
Killing |
Stopping container kserve-container | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-617e5 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e3b29-predictor-7b4b4f756d-mzxqg |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e3b29-predictor-596b8c9d77-b79mj |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-d2d49-predictor-c9cf5cd84 |
SuccessfulCreate |
Created pod: success-200-isvc-d2d49-predictor-c9cf5cd84-q4dlk | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e3b29-predictor-7b4b4f756d-mzxqg |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-d2d49-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-d2d49-predictor-c9cf5cd84 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d2d49-predictor-c9cf5cd84-q4dlk |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d2d49-predictor-9686f48b9-cnvdw |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-d2d49-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
multus |
success-200-isvc-d2d49-predictor-c9cf5cd84-q4dlk |
AddedInterface |
Add eth0 [10.134.0.36/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d2d49-predictor-c9cf5cd84-q4dlk |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1987" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d2d49-predictor-c9cf5cd84-q4dlk |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d2d49-predictor-c9cf5cd84-q4dlk |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d2d49-predictor-c9cf5cd84-q4dlk |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d2d49-predictor-c9cf5cd84-q4dlk |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d2d49-predictor-9686f48b9-cnvdw |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1987" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e3b29-predictor-596b8c9d77-b79mj |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.22:8643/healthz": dial tcp 10.133.0.22:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d2d49-predictor-9686f48b9-cnvdw |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e3b29-predictor-7b4b4f756d-mzxqg |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.34:8643/healthz": dial tcp 10.134.0.34:8643: connect: connection refused | |
kserve-ci-e2e-test |
multus |
error-404-isvc-d2d49-predictor-9686f48b9-cnvdw |
AddedInterface |
Add eth0 [10.133.0.24/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d2d49-predictor-9686f48b9-cnvdw |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d2d49-predictor-9686f48b9-cnvdw |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d2d49-predictor-9686f48b9-cnvdw |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d2d49-predictor-9686f48b9-cnvdw |
Created |
Created container: kube-rbac-proxy | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-e3b29-predictor-7b4b4f756d-mzxqg |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.34:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-e3b29-predictor-596b8c9d77-b79mj |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.22:8080: connect: connection refused |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-617e5-69d5f987b8 |
SuccessfulCreate |
Created pod: splitter-graph-617e5-69d5f987b8-qz8dk | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-617e5 |
ScalingReplicaSet |
Scaled up replica set splitter-graph-617e5-69d5f987b8 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-617e5 |
InferenceGraphReady |
InferenceGraph [splitter-graph-617e5] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-617e5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-617e5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-617e5 |
UpdateFailed |
Failed to update status for InferenceGraph "splitter-graph-617e5": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "splitter-graph-617e5": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-617e5 |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "splitter-graph-617e5": the object has been modified; please apply your changes to the latest version and try again | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-617e5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-617e5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
multus |
splitter-graph-617e5-69d5f987b8-qz8dk |
AddedInterface |
Add eth0 [10.132.0.41/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-617e5-69d5f987b8-qz8dk |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:8bcf19d9ed90070ea776e0c3189bb53a4b1362b2d66bf974f83dca260544e4db" already present on machine | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-617e5-69d5f987b8-qz8dk |
Started |
Started container splitter-graph-617e5 | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-617e5-69d5f987b8-qz8dk |
Created |
Created container: splitter-graph-617e5 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-617e5-predictor-776d98c8-5r9t7 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-617e5-predictor-776d98c8-5r9t7 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-617e5-69d5f987b8-qz8dk |
Killing |
Stopping container splitter-graph-617e5 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-617e5-predictor-6dd67cff67-zcf86 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-617e5-predictor-6dd67cff67-zcf86 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0e2c5-predictor-7956b5b96-jwqzd |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-0e2c5 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-0e2c5": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-0e2c5": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-0e2c5-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-0e2c5-predictor-7956b5b96 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-0e2c5-predictor-7956b5b96 |
SuccessfulCreate |
Created pod: success-200-isvc-0e2c5-predictor-7956b5b96-jwqzd | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0e2c5-predictor-7956b5b96-jwqzd |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0e2c5-predictor-7956b5b96-jwqzd |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0e2c5-predictor-7956b5b96-jwqzd |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0e2c5-predictor-7956b5b96-jwqzd |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0e2c5-predictor-7956b5b96-jwqzd |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1987" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-0e2c5-predictor-7956b5b96-jwqzd |
AddedInterface |
Add eth0 [10.134.0.37/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-0e2c5-predictor-7cb46d9cfc |
SuccessfulCreate |
Created pod: error-404-isvc-0e2c5-predictor-7cb46d9cfc-7f2gl | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-0e2c5 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-0e2c5": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-0e2c5-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-0e2c5-predictor-7cb46d9cfc from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0e2c5-predictor-7cb46d9cfc-7f2gl |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0e2c5-predictor-7cb46d9cfc-7f2gl |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0e2c5-predictor-7cb46d9cfc-7f2gl |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0e2c5-predictor-7cb46d9cfc-7f2gl |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0e2c5-predictor-7cb46d9cfc-7f2gl |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0e2c5-predictor-7cb46d9cfc-7f2gl |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1987" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-0e2c5-predictor-7cb46d9cfc-7f2gl |
AddedInterface |
Add eth0 [10.133.0.25/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-617e5-predictor-776d98c8-5r9t7 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.35:8643/healthz": dial tcp 10.134.0.35:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-e3b29-879b5696f-mrtph |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d2d49-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d2d49-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-d2d49-predictor-c9cf5cd84-q4dlk |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.36:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-d2d49-predictor-9686f48b9-cnvdw |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.24:8080: connect: connection refused |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d2d49-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d2d49-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d2d49 |
InferenceServiceReady |
InferenceService [success-200-isvc-d2d49] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d2d49 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-617e5-69d5f987b8-qz8dk |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d2d49 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d2d49 |
InferenceServiceReady |
InferenceService [error-404-isvc-d2d49] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-0e2c5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-0e2c5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-0e2c5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-0e2c5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-d2d49 |
ScalingReplicaSet |
Scaled up replica set switch-graph-d2d49-6548fd667c from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-d2d49 |
InferenceGraphReady |
InferenceGraph [switch-graph-d2d49] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-d2d49-6548fd667c |
SuccessfulCreate |
Created pod: switch-graph-d2d49-6548fd667c-bsphk | |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-0e2c5-predictor-7cb46d9cfc-7f2gl |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.25:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
switch-graph-d2d49-6548fd667c-bsphk |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-d2d49-serving-cert" not found | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-d2d49 |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "switch-graph-d2d49": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-d2d49 |
UpdateFailed |
Failed to update status for InferenceGraph "switch-graph-d2d49": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "switch-graph-d2d49": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
switch-graph-d2d49-6548fd667c-bsphk |
Created |
Created container: switch-graph-d2d49 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-d2d49-6548fd667c-bsphk |
Started |
Started container switch-graph-d2d49 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-d2d49-6548fd667c-bsphk |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:8bcf19d9ed90070ea776e0c3189bb53a4b1362b2d66bf974f83dca260544e4db" already present on machine | |
kserve-ci-e2e-test |
multus |
switch-graph-d2d49-6548fd667c-bsphk |
AddedInterface |
Add eth0 [10.132.0.42/23] from ovn-kubernetes | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-0e2c5 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-0e2c5 |
InferenceServiceReady |
InferenceService [success-200-isvc-0e2c5] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-0e2c5 |
InferenceServiceReady |
InferenceService [error-404-isvc-0e2c5] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-0e2c5 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d2d49-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d2d49-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-d2d49 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-d2d49 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-0e2c5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
splitter-graph-0e2c5-5df677ff7b-cvghx |
Started |
Started container splitter-graph-0e2c5 | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-0e2c5-5df677ff7b |
SuccessfulCreate |
Created pod: splitter-graph-0e2c5-5df677ff7b-cvghx | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-0e2c5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
multus |
splitter-graph-0e2c5-5df677ff7b-cvghx |
AddedInterface |
Add eth0 [10.132.0.43/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-0e2c5 |
ScalingReplicaSet |
Scaled up replica set splitter-graph-0e2c5-5df677ff7b from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-0e2c5-5df677ff7b-cvghx |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:8bcf19d9ed90070ea776e0c3189bb53a4b1362b2d66bf974f83dca260544e4db" already present on machine | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-0e2c5-5df677ff7b-cvghx |
Created |
Created container: splitter-graph-0e2c5 | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-0e2c5 |
InferenceGraphReady |
InferenceGraph [splitter-graph-0e2c5] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-0e2c5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-0e2c5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-d2d49 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-d2d49 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-0e2c5 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-0e2c5 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d2d49-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d2d49-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0e2c5-predictor-7956b5b96-jwqzd |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0e2c5-predictor-7cb46d9cfc-7f2gl |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-0e2c5-predictor-7cb46d9cfc-7f2gl |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0e2c5-predictor-7956b5b96-jwqzd |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-0e2c5-5df677ff7b-cvghx |
Killing |
Stopping container splitter-graph-0e2c5 | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-0e2c5-predictor-7956b5b96-jwqzd |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.37:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-0e2c5-predictor-7956b5b96-jwqzd |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.37:8643/healthz": dial tcp 10.134.0.37:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-0e2c5-5df677ff7b-cvghx |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d2d49-predictor-9686f48b9-cnvdw |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d2d49-predictor-9686f48b9-cnvdw |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
switch-graph-d2d49-6548fd667c-bsphk |
Killing |
Stopping container switch-graph-d2d49 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d2d49-predictor-c9cf5cd84-q4dlk |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d2d49-predictor-c9cf5cd84-q4dlk |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d2d49-predictor-c9cf5cd84-q4dlk |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.36:8643/healthz": dial tcp 10.134.0.36:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-d2d49-6548fd667c-bsphk |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |