| Time | Namespace | Component | RelatedObject | Reason | Message |
|---|---|---|---|---|---|
kserve-ci-e2e-test |
success-200-isvc-c2858-predictor-d76bb676c-2vf88 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-c2858-predictor-d76bb676c-2vf88 to ip-10-0-143-228.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-2fdf6-7487d76957-m5hkw |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-2fdf6-7487d76957-m5hkw to ip-10-0-143-112.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-93f48-predictor-7bb978bc98-f8kx7 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-93f48-predictor-7bb978bc98-f8kx7 to ip-10-0-143-112.ec2.internal | ||
kserve-ci-e2e-test |
isvc-xgboost-graph-predictor-555fc69d56-st7bs |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-st7bs to ip-10-0-142-146.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-eb976-predictor-d669575b4-j4rjg |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-eb976-predictor-d669575b4-j4rjg to ip-10-0-143-112.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-b5187-predictor-d99857bc6-97v66 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-b5187-predictor-d99857bc6-97v66 to ip-10-0-143-112.ec2.internal | ||
kserve-ci-e2e-test |
model-chainer-5f85d95997-z2skz |
Scheduled |
Successfully assigned kserve-ci-e2e-test/model-chainer-5f85d95997-z2skz to ip-10-0-143-112.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-93f48-78c9465b4b-nx7ln |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-93f48-78c9465b4b-nx7ln to ip-10-0-143-112.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-2e2a5-848d4d594c-z5css |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-2e2a5-848d4d594c-z5css to ip-10-0-143-112.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-c2858-54876bd879-lh79j |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-c2858-54876bd879-lh79j to ip-10-0-143-112.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-d52cc-6b7fc7b8b9-nfzxq |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-d52cc-6b7fc7b8b9-nfzxq to ip-10-0-143-112.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-f3946-5c566544b8-gg6fs |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-f3946-5c566544b8-gg6fs to ip-10-0-143-112.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-2fdf6-predictor-fff5f778b-5tld6 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-2fdf6-predictor-fff5f778b-5tld6 to ip-10-0-143-112.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-2756e-6c865d8977-c2ssw |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-2756e-6c865d8977-c2ssw to ip-10-0-143-112.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-eb976-84f6d675cc-txp68 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-eb976-84f6d675cc-txp68 to ip-10-0-143-112.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-b5187-58856df79d-g7x99 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-b5187-58856df79d-g7x99 to ip-10-0-143-112.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-f3946-predictor-8f678845f-54nlw |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-f3946-predictor-8f678845f-54nlw to ip-10-0-143-228.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-f0d22-predictor-69c49986b7-kp9jw |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-f0d22-predictor-69c49986b7-kp9jw to ip-10-0-143-228.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-f0d22-predictor-7667c7cb86-cqmgb |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-f0d22-predictor-7667c7cb86-cqmgb to ip-10-0-143-112.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-d52cc-predictor-65bf9c9c88-b7k7x |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-d52cc-predictor-65bf9c9c88-b7k7x to ip-10-0-143-112.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-2e2a5-predictor-65f657f664-rbwl2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-2e2a5-predictor-65f657f664-rbwl2 to ip-10-0-143-112.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-1-predictor-75998957b8-hsmvn |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75998957b8-hsmvn to ip-10-0-143-228.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-2756e-predictor-58d849df88-k4hdz |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-2756e-predictor-58d849df88-k4hdz to ip-10-0-143-112.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-d52cc-predictor-5cb644f657-6t22c |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-d52cc-predictor-5cb644f657-6t22c to ip-10-0-143-228.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-2756e-predictor-5c99844f45-rhfvz |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-2756e-predictor-5c99844f45-rhfvz to ip-10-0-143-228.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-2e2a5-predictor-5cd7c778fc-rqd7n |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-2e2a5-predictor-5cd7c778fc-rqd7n to ip-10-0-143-228.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-2fdf6-predictor-74959c8f6c-t8vgn |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-2fdf6-predictor-74959c8f6c-t8vgn to ip-10-0-143-228.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-93f48-predictor-695cd6968c-m8g6m |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-93f48-predictor-695cd6968c-m8g6m to ip-10-0-143-228.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-b5187-predictor-59c96b8b97-nzkvb |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-b5187-predictor-59c96b8b97-nzkvb to ip-10-0-143-228.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-2-predictor-7756fccfc-8b9k7 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7756fccfc-8b9k7 to ip-10-0-142-146.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-c2858-predictor-69ddfb9f7d-bfxf9 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-c2858-predictor-69ddfb9f7d-bfxf9 to ip-10-0-143-112.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-f0d22-6c68dfbc6f-dmg57 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-f0d22-6c68dfbc6f-dmg57 to ip-10-0-143-112.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-f3946-predictor-c448bf9d-q7xtf |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-f3946-predictor-c448bf9d-q7xtf to ip-10-0-143-112.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-eb976-predictor-5c8648d4bf-v8nkx |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-eb976-predictor-5c8648d4bf-v8nkx to ip-10-0-143-228.ec2.internal | ||
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-1-predictor-75998957b8 |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-1-predictor-75998957b8-hsmvn | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-75998957b8-hsmvn |
Pulling |
Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:8abde4da7e87a625b928a82fb491989fd006b5b2f5e4745e1871a977e20e0f2e" | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-2-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-2-predictor-7756fccfc from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-2-predictor-7756fccfc |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-2-predictor-7756fccfc-8b9k7 | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-xgboost-graph-predictor-555fc69d56 |
SuccessfulCreate |
Created pod: isvc-xgboost-graph-predictor-555fc69d56-st7bs | |
kserve-ci-e2e-test |
deployment-controller |
isvc-xgboost-graph-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-xgboost-graph-predictor-555fc69d56 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-1-predictor-75998957b8-hsmvn |
AddedInterface |
Add eth0 [10.134.0.21/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-eb976-predictor-d669575b4 |
SuccessfulCreate |
Created pod: error-404-isvc-eb976-predictor-d669575b4-j4rjg | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-eb976-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-eb976-predictor-5c8648d4bf from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-eb976-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-eb976-predictor-d669575b4 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-eb976-predictor-5c8648d4bf |
SuccessfulCreate |
Created pod: success-200-isvc-eb976-predictor-5c8648d4bf-v8nkx | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-1-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-1-predictor-75998957b8 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7756fccfc-8b9k7 |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-eb976-predictor-5c8648d4bf-v8nkx |
Pulling |
Pulling image "quay.io/opendatahub/success-200-isvc:odh-pr-1955" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-eb976-predictor-d669575b4-j4rjg |
Pulling |
Pulling image "quay.io/opendatahub/error-404-isvc:odh-pr-1955" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-st7bs |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-st7bs |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-st7bs |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:8abde4da7e87a625b928a82fb491989fd006b5b2f5e4745e1871a977e20e0f2e" already present on machine | |
kserve-ci-e2e-test |
multus |
isvc-xgboost-graph-predictor-555fc69d56-st7bs |
AddedInterface |
Add eth0 [10.132.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-2-predictor-7756fccfc-8b9k7 |
AddedInterface |
Add eth0 [10.132.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
success-200-isvc-eb976-predictor-5c8648d4bf-v8nkx |
AddedInterface |
Add eth0 [10.134.0.22/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7756fccfc-8b9k7 |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:8abde4da7e87a625b928a82fb491989fd006b5b2f5e4745e1871a977e20e0f2e" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-eb976-predictor-d669575b4-j4rjg |
AddedInterface |
Add eth0 [10.133.0.24/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7756fccfc-8b9k7 |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-st7bs |
Pulling |
Pulling image "kserve/xgbserver:latest" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7756fccfc-8b9k7 |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1955" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-eb976-predictor-d669575b4-j4rjg |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-eb976-predictor-d669575b4-j4rjg |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-eb976-predictor-d669575b4-j4rjg |
Pulled |
Successfully pulled image "quay.io/opendatahub/error-404-isvc:odh-pr-1955" in 11.306s (11.306s including waiting). Image size: 1355791443 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-eb976-predictor-d669575b4-j4rjg |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-75998957b8-hsmvn |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:8abde4da7e87a625b928a82fb491989fd006b5b2f5e4745e1871a977e20e0f2e" in 13.637s (13.637s including waiting). Image size: 306673092 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-eb976-predictor-5c8648d4bf-v8nkx |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-eb976-predictor-d669575b4-j4rjg |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.194s (2.194s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-eb976-predictor-d669575b4-j4rjg |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-eb976-predictor-5c8648d4bf-v8nkx |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-eb976-predictor-5c8648d4bf-v8nkx |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-eb976-predictor-5c8648d4bf-v8nkx |
Pulled |
Successfully pulled image "quay.io/opendatahub/success-200-isvc:odh-pr-1955" in 13.55s (13.55s including waiting). Image size: 1356786777 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-75998957b8-hsmvn |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-eb976-predictor-d669575b4-j4rjg |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-75998957b8-hsmvn |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-eb976-predictor-5c8648d4bf-v8nkx |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-eb976-predictor-5c8648d4bf-v8nkx |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.307s (2.307s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-eb976-predictor-5c8648d4bf-v8nkx |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-75998957b8-hsmvn |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1955" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-75998957b8-hsmvn |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1955" in 6.059s (6.059s including waiting). Image size: 1580728223 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-75998957b8-hsmvn |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-75998957b8-hsmvn |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-75998957b8-hsmvn |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-75998957b8-hsmvn |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-75998957b8-hsmvn |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-st7bs |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-st7bs |
Pulled |
Successfully pulled image "kserve/xgbserver:latest" in 26.257s (26.257s including waiting). Image size: 1435742979 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-st7bs |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-st7bs |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7756fccfc-8b9k7 |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7756fccfc-8b9k7 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7756fccfc-8b9k7 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7756fccfc-8b9k7 |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1955" in 24.801s (24.801s including waiting). Image size: 1580728223 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-st7bs |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.065s (2.065s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-st7bs |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-st7bs |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7756fccfc-8b9k7 |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.469s (2.469s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7756fccfc-8b9k7 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7756fccfc-8b9k7 |
Created |
Created container: kube-rbac-proxy | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-eb976-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-eb976-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-eb976-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-eb976-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-eb976-predictor-5c8648d4bf-v8nkx |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.22:8080: connect: connection refused |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-eb976 |
InferenceServiceReady |
InferenceService [error-404-isvc-eb976] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-eb976 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-eb976 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-eb976 |
InferenceServiceReady |
InferenceService [success-200-isvc-eb976] is Ready | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-eb976-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-eb976-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
switch-graph-eb976-84f6d675cc-txp68 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-eb976-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-eb976-84f6d675cc |
SuccessfulCreate |
Created pod: switch-graph-eb976-84f6d675cc-txp68 | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-eb976 |
ScalingReplicaSet |
Scaled up replica set switch-graph-eb976-84f6d675cc from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-eb976 |
InferenceGraphReady |
InferenceGraph [switch-graph-eb976] is Ready | |
kserve-ci-e2e-test |
kubelet |
switch-graph-eb976-84f6d675cc-txp68 |
Pulling |
Pulling image "quay.io/opendatahub/kserve-router@sha256:dae09a3aa9a23aca96ea2af0400a27c12957dcc91259106b40cbd871a02ad5c3" | |
kserve-ci-e2e-test |
multus |
switch-graph-eb976-84f6d675cc-txp68 |
AddedInterface |
Add eth0 [10.133.0.25/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-eb976-84f6d675cc-txp68 |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-router@sha256:dae09a3aa9a23aca96ea2af0400a27c12957dcc91259106b40cbd871a02ad5c3" in 2.69s (2.69s including waiting). Image size: 211751527 bytes. | |
kserve-ci-e2e-test |
kubelet |
switch-graph-eb976-84f6d675cc-txp68 |
Created |
Created container: switch-graph-eb976 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-eb976-84f6d675cc-txp68 |
Started |
Started container switch-graph-eb976 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-eb976-predictor-5c8648d4bf-v8nkx |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-eb976-predictor-d669575b4-j4rjg |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
switch-graph-eb976-84f6d675cc-txp68 |
Killing |
Stopping container switch-graph-eb976 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-b5187-predictor-d99857bc6 |
SuccessfulCreate |
Created pod: error-404-isvc-b5187-predictor-d99857bc6-97v66 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-b5187-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-b5187-predictor-d99857bc6 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-eb976-predictor-d669575b4-j4rjg |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b5187-predictor-59c96b8b97-nzkvb |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-b5187-predictor-serving-cert" not found | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-eb976-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-eb976-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x8) | kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-st7bs |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.31:8080: connect: connection refused |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-b5187-predictor-59c96b8b97 |
SuccessfulCreate |
Created pod: success-200-isvc-b5187-predictor-59c96b8b97-nzkvb | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-b5187-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-b5187-predictor-59c96b8b97 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-eb976-predictor-5c8648d4bf-v8nkx |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
multus |
success-200-isvc-b5187-predictor-59c96b8b97-nzkvb |
AddedInterface |
Add eth0 [10.134.0.23/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-eb976-predictor-d669575b4-j4rjg |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.24:8643/healthz": dial tcp 10.133.0.24:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b5187-predictor-d99857bc6-97v66 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-b5187-predictor-serving-cert" not found | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-eb976-predictor-d669575b4-j4rjg |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.24:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b5187-predictor-59c96b8b97-nzkvb |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1955" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b5187-predictor-59c96b8b97-nzkvb |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b5187-predictor-d99857bc6-97v66 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b5187-predictor-d99857bc6-97v66 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b5187-predictor-d99857bc6-97v66 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1955" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b5187-predictor-59c96b8b97-nzkvb |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b5187-predictor-59c96b8b97-nzkvb |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b5187-predictor-59c96b8b97-nzkvb |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b5187-predictor-59c96b8b97-nzkvb |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b5187-predictor-d99857bc6-97v66 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-b5187-predictor-d99857bc6-97v66 |
AddedInterface |
Add eth0 [10.133.0.26/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b5187-predictor-d99857bc6-97v66 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b5187-predictor-d99857bc6-97v66 |
Started |
Started container kube-rbac-proxy | |
| (x10) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-75998957b8-hsmvn |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.21:8080: connect: connection refused |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
InferenceServiceReady |
InferenceService [isvc-xgboost-graph] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-1] is Ready | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-2] is Ready | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-eb976-84f6d675cc-txp68 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-b5187-predictor-59c96b8b97-nzkvb |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.23:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-b5187-predictor-d99857bc6-97v66 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.26:8080: connect: connection refused |
kserve-ci-e2e-test |
deployment-controller |
model-chainer |
ScalingReplicaSet |
Scaled up replica set model-chainer-5f85d95997 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
model-chainer-5f85d95997-z2skz |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "model-chainer-serving-cert" not found | |
kserve-ci-e2e-test |
InferenceGraphController |
model-chainer |
InferenceGraphReady |
InferenceGraph [model-chainer] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
model-chainer-5f85d95997 |
SuccessfulCreate |
Created pod: model-chainer-5f85d95997-z2skz | |
kserve-ci-e2e-test |
kubelet |
model-chainer-5f85d95997-z2skz |
Started |
Started container model-chainer | |
kserve-ci-e2e-test |
multus |
model-chainer-5f85d95997-z2skz |
AddedInterface |
Add eth0 [10.133.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
model-chainer-5f85d95997-z2skz |
Created |
Created container: model-chainer | |
kserve-ci-e2e-test |
kubelet |
model-chainer-5f85d95997-z2skz |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:dae09a3aa9a23aca96ea2af0400a27c12957dcc91259106b40cbd871a02ad5c3" already present on machine | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-b5187-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-b5187-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-b5187-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-b5187-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-b5187 |
InferenceServiceReady |
InferenceService [success-200-isvc-b5187] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-b5187 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-b5187 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-b5187 |
InferenceServiceReady |
InferenceService [error-404-isvc-b5187] is Ready | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-75998957b8-hsmvn |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-st7bs |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7756fccfc-8b9k7 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7756fccfc-8b9k7 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
model-chainer-5f85d95997-z2skz |
Killing |
Stopping container model-chainer | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-st7bs |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-75998957b8-hsmvn |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-f3946-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-f3946-predictor-8f678845f from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-f3946-predictor-8f678845f |
SuccessfulCreate |
Created pod: success-200-isvc-f3946-predictor-8f678845f-54nlw | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7756fccfc-8b9k7 |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.32:8643/healthz": dial tcp 10.132.0.32:8643: connect: connection refused | |
| (x10) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-7756fccfc-8b9k7 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.32:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f3946-predictor-c448bf9d-q7xtf |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-f3946-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-75998957b8-hsmvn |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.21:8643/healthz": dial tcp 10.134.0.21:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f3946-predictor-8f678845f-54nlw |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-f3946-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-f3946-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-f3946-predictor-c448bf9d from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-f3946-predictor-c448bf9d |
SuccessfulCreate |
Created pod: error-404-isvc-f3946-predictor-c448bf9d-q7xtf | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f3946-predictor-c448bf9d-q7xtf |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1955" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f3946-predictor-c448bf9d-q7xtf |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f3946-predictor-8f678845f-54nlw |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1955" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-f3946-predictor-8f678845f-54nlw |
AddedInterface |
Add eth0 [10.134.0.24/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f3946-predictor-8f678845f-54nlw |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-f3946-predictor-c448bf9d-q7xtf |
AddedInterface |
Add eth0 [10.133.0.28/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f3946-predictor-8f678845f-54nlw |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f3946-predictor-c448bf9d-q7xtf |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f3946-predictor-c448bf9d-q7xtf |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f3946-predictor-8f678845f-54nlw |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f3946-predictor-c448bf9d-q7xtf |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f3946-predictor-c448bf9d-q7xtf |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f3946-predictor-8f678845f-54nlw |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f3946-predictor-8f678845f-54nlw |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-b5187 |
ScalingReplicaSet |
Scaled up replica set switch-graph-b5187-58856df79d from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-b5187 |
InferenceGraphReady |
InferenceGraph [switch-graph-b5187] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-b5187-58856df79d |
SuccessfulCreate |
Created pod: switch-graph-b5187-58856df79d-g7x99 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-b5187-58856df79d-g7x99 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:dae09a3aa9a23aca96ea2af0400a27c12957dcc91259106b40cbd871a02ad5c3" already present on machine | |
kserve-ci-e2e-test |
multus |
switch-graph-b5187-58856df79d-g7x99 |
AddedInterface |
Add eth0 [10.133.0.29/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-b5187-58856df79d-g7x99 |
Created |
Created container: switch-graph-b5187 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-b5187-58856df79d-g7x99 |
Started |
Started container switch-graph-b5187 | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-b5187-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-b5187-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-b5187-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-b5187-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
model-chainer-5f85d95997-z2skz |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f3946-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f3946-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f3946-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f3946-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-f3946-predictor-c448bf9d-q7xtf |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.28:8080: connect: connection refused |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-b5187 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-b5187 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-f3946 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-f3946 |
InferenceServiceReady |
InferenceService [success-200-isvc-f3946] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f3946 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f3946 |
InferenceServiceReady |
InferenceService [error-404-isvc-f3946] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-f3946 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-f3946-5c566544b8 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-f3946-5c566544b8 |
SuccessfulCreate |
Created pod: sequence-graph-f3946-5c566544b8-gg6fs | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f3946-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-f3946 |
InferenceGraphReady |
InferenceGraph [sequence-graph-f3946] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f3946-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
multus |
sequence-graph-f3946-5c566544b8-gg6fs |
AddedInterface |
Add eth0 [10.133.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-f3946-5c566544b8-gg6fs |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:dae09a3aa9a23aca96ea2af0400a27c12957dcc91259106b40cbd871a02ad5c3" already present on machine | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f3946-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
sequence-graph-f3946-5c566544b8-gg6fs |
Created |
Created container: sequence-graph-f3946 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-f3946-5c566544b8-gg6fs |
Started |
Started container sequence-graph-f3946 | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f3946-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-f3946 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-f3946 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b5187-predictor-59c96b8b97-nzkvb |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-93f48-predictor-695cd6968c-m8g6m |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-93f48-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-b5187-predictor-59c96b8b97-nzkvb |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-93f48-predictor-7bb978bc98-f8kx7 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-93f48-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b5187-predictor-d99857bc6-97v66 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-b5187-predictor-d99857bc6-97v66 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-93f48-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-93f48-predictor-695cd6968c from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-93f48-predictor-7bb978bc98 |
SuccessfulCreate |
Created pod: error-404-isvc-93f48-predictor-7bb978bc98-f8kx7 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-93f48-predictor-695cd6968c |
SuccessfulCreate |
Created pod: success-200-isvc-93f48-predictor-695cd6968c-m8g6m | |
kserve-ci-e2e-test |
kubelet |
switch-graph-b5187-58856df79d-g7x99 |
Killing |
Stopping container switch-graph-b5187 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-93f48-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-93f48-predictor-7bb978bc98 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
success-200-isvc-93f48-predictor-695cd6968c-m8g6m |
AddedInterface |
Add eth0 [10.134.0.25/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-93f48-predictor-695cd6968c-m8g6m |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-93f48-predictor-695cd6968c-m8g6m |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1955" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-93f48-predictor-695cd6968c-m8g6m |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-93f48-predictor-695cd6968c-m8g6m |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-93f48-predictor-695cd6968c-m8g6m |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-93f48-predictor-695cd6968c-m8g6m |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-93f48-predictor-7bb978bc98-f8kx7 |
AddedInterface |
Add eth0 [10.133.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-93f48-predictor-7bb978bc98-f8kx7 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1955" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-93f48-predictor-7bb978bc98-f8kx7 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-93f48-predictor-7bb978bc98-f8kx7 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-93f48-predictor-7bb978bc98-f8kx7 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-93f48-predictor-7bb978bc98-f8kx7 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-93f48-predictor-7bb978bc98-f8kx7 |
Started |
Started container kserve-container | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-b5187-58856df79d-g7x99 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-93f48-predictor-695cd6968c-m8g6m |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.25:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-93f48-predictor-7bb978bc98-f8kx7 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.31:8080: connect: connection refused |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-93f48-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-93f48-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-93f48 |
InferenceServiceReady |
InferenceService [success-200-isvc-93f48] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-93f48 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-93f48 |
InferenceServiceReady |
InferenceService [error-404-isvc-93f48] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-93f48 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f3946-predictor-c448bf9d-q7xtf |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-f3946-5c566544b8-gg6fs |
Killing |
Stopping container sequence-graph-f3946 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f3946-predictor-8f678845f-54nlw |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-2e2a5 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-2e2a5": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-2e2a5": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-2e2a5-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-2e2a5-predictor-5cd7c778fc from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-2e2a5-predictor-65f657f664 |
SuccessfulCreate |
Created pod: error-404-isvc-2e2a5-predictor-65f657f664-rbwl2 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f3946-predictor-8f678845f-54nlw |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-2e2a5 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-2e2a5": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-2e2a5 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-2e2a5": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-2e2a5-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-2e2a5-predictor-65f657f664 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f3946-predictor-c448bf9d-q7xtf |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-2e2a5-predictor-5cd7c778fc |
SuccessfulCreate |
Created pod: success-200-isvc-2e2a5-predictor-5cd7c778fc-rqd7n | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e2a5-predictor-5cd7c778fc-rqd7n |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-2e2a5-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-2e2a5 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-2e2a5": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-2e2a5": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
multus |
error-404-isvc-2e2a5-predictor-65f657f664-rbwl2 |
AddedInterface |
Add eth0 [10.133.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e2a5-predictor-65f657f664-rbwl2 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e2a5-predictor-65f657f664-rbwl2 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e2a5-predictor-65f657f664-rbwl2 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1955" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e2a5-predictor-65f657f664-rbwl2 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e2a5-predictor-65f657f664-rbwl2 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e2a5-predictor-65f657f664-rbwl2 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e2a5-predictor-5cd7c778fc-rqd7n |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e2a5-predictor-5cd7c778fc-rqd7n |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-2e2a5-predictor-5cd7c778fc-rqd7n |
AddedInterface |
Add eth0 [10.134.0.26/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e2a5-predictor-5cd7c778fc-rqd7n |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1955" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e2a5-predictor-5cd7c778fc-rqd7n |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e2a5-predictor-5cd7c778fc-rqd7n |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e2a5-predictor-5cd7c778fc-rqd7n |
Started |
Started container kube-rbac-proxy | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-f3946-predictor-8f678845f-54nlw |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.24:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f3946-predictor-8f678845f-54nlw |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.24:8643/healthz": dial tcp 10.134.0.24:8643: connect: connection refused | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-93f48-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-93f48 |
InferenceGraphReady |
InferenceGraph [ensemble-graph-93f48] is Ready | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-93f48-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-93f48-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-93f48-78c9465b4b |
SuccessfulCreate |
Created pod: ensemble-graph-93f48-78c9465b4b-nx7ln | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-93f48 |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-93f48-78c9465b4b from 0 to 1 | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-93f48-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-93f48-78c9465b4b-nx7ln |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:dae09a3aa9a23aca96ea2af0400a27c12957dcc91259106b40cbd871a02ad5c3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-93f48-78c9465b4b-nx7ln |
Started |
Started container ensemble-graph-93f48 | |
kserve-ci-e2e-test |
multus |
ensemble-graph-93f48-78c9465b4b-nx7ln |
AddedInterface |
Add eth0 [10.133.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-93f48-78c9465b4b-nx7ln |
Created |
Created container: ensemble-graph-93f48 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-93f48-predictor-695cd6968c-m8g6m |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-93f48-78c9465b4b-nx7ln |
Killing |
Stopping container ensemble-graph-93f48 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-93f48-predictor-695cd6968c-m8g6m |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-93f48-predictor-7bb978bc98-f8kx7 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-93f48-predictor-7bb978bc98-f8kx7 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d52cc-predictor-5cb644f657-6t22c |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d52cc-predictor-5cb644f657-6t22c |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d52cc-predictor-65bf9c9c88-b7k7x |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-d52cc-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d52cc-predictor-5cb644f657-6t22c |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d52cc-predictor-5cb644f657-6t22c |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-d52cc-predictor-5cb644f657 |
SuccessfulCreate |
Created pod: success-200-isvc-d52cc-predictor-5cb644f657-6t22c | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-d52cc-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-d52cc-predictor-65bf9c9c88 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-d52cc-predictor-65bf9c9c88 |
SuccessfulCreate |
Created pod: error-404-isvc-d52cc-predictor-65bf9c9c88-b7k7x | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-d52cc-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-d52cc-predictor-5cb644f657 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
success-200-isvc-d52cc-predictor-5cb644f657-6t22c |
AddedInterface |
Add eth0 [10.134.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d52cc-predictor-5cb644f657-6t22c |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d52cc-predictor-5cb644f657-6t22c |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1955" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d52cc-predictor-65bf9c9c88-b7k7x |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1955" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d52cc-predictor-65bf9c9c88-b7k7x |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-d52cc-predictor-65bf9c9c88-b7k7x |
AddedInterface |
Add eth0 [10.133.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d52cc-predictor-65bf9c9c88-b7k7x |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d52cc-predictor-65bf9c9c88-b7k7x |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d52cc-predictor-65bf9c9c88-b7k7x |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d52cc-predictor-65bf9c9c88-b7k7x |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-93f48-predictor-695cd6968c-m8g6m |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.25:8643/healthz": dial tcp 10.134.0.25:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-f3946-5c566544b8-gg6fs |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2e2a5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2e2a5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2e2a5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2e2a5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e2a5-predictor-5cd7c778fc-rqd7n |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.26:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e2a5-predictor-65f657f664-rbwl2 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.32:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-93f48-78c9465b4b-nx7ln |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-2e2a5 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-2e2a5 |
InferenceServiceReady |
InferenceService [success-200-isvc-2e2a5] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-2e2a5 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-2e2a5 |
InferenceServiceReady |
InferenceService [error-404-isvc-2e2a5] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d52cc-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d52cc-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2e2a5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2e2a5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-2e2a5-848d4d594c |
SuccessfulCreate |
Created pod: sequence-graph-2e2a5-848d4d594c-z5css | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2e2a5-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2e2a5-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-2e2a5 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-2e2a5-848d4d594c from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-2e2a5 |
InferenceGraphReady |
InferenceGraph [sequence-graph-2e2a5] is Ready | |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-d52cc-predictor-65bf9c9c88-b7k7x |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.34:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
sequence-graph-2e2a5-848d4d594c-z5css |
Started |
Started container sequence-graph-2e2a5 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-2e2a5-848d4d594c-z5css |
Created |
Created container: sequence-graph-2e2a5 | |
kserve-ci-e2e-test |
multus |
sequence-graph-2e2a5-848d4d594c-z5css |
AddedInterface |
Add eth0 [10.133.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-2e2a5-848d4d594c-z5css |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:dae09a3aa9a23aca96ea2af0400a27c12957dcc91259106b40cbd871a02ad5c3" already present on machine | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d52cc-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d52cc-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d52cc |
InferenceServiceReady |
InferenceService [success-200-isvc-d52cc] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-d52cc |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
kubelet |
sequence-graph-2e2a5-848d4d594c-z5css |
Killing |
Stopping container sequence-graph-2e2a5 | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d52cc |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-d52cc |
InferenceServiceReady |
InferenceService [error-404-isvc-d52cc] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-c2858-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-c2858-predictor-69ddfb9f7d from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e2a5-predictor-5cd7c778fc-rqd7n |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e2a5-predictor-5cd7c778fc-rqd7n |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e2a5-predictor-65f657f664-rbwl2 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2e2a5-predictor-65f657f664-rbwl2 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-c2858-predictor-69ddfb9f7d |
SuccessfulCreate |
Created pod: error-404-isvc-c2858-predictor-69ddfb9f7d-bfxf9 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c2858-predictor-d76bb676c-2vf88 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-c2858-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c2858-predictor-69ddfb9f7d-bfxf9 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-c2858-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-c2858-predictor-d76bb676c |
SuccessfulCreate |
Created pod: success-200-isvc-c2858-predictor-d76bb676c-2vf88 | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-c2858-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-c2858-predictor-d76bb676c from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c2858-predictor-d76bb676c-2vf88 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1955" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-c2858-predictor-d76bb676c-2vf88 |
AddedInterface |
Add eth0 [10.134.0.28/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c2858-predictor-69ddfb9f7d-bfxf9 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1955" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c2858-predictor-d76bb676c-2vf88 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c2858-predictor-69ddfb9f7d-bfxf9 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c2858-predictor-d76bb676c-2vf88 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c2858-predictor-69ddfb9f7d-bfxf9 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c2858-predictor-69ddfb9f7d-bfxf9 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
error-404-isvc-c2858-predictor-69ddfb9f7d-bfxf9 |
AddedInterface |
Add eth0 [10.133.0.36/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c2858-predictor-69ddfb9f7d-bfxf9 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c2858-predictor-69ddfb9f7d-bfxf9 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c2858-predictor-d76bb676c-2vf88 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c2858-predictor-d76bb676c-2vf88 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c2858-predictor-d76bb676c-2vf88 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2e2a5-predictor-5cd7c778fc-rqd7n |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.26:8643/healthz": dial tcp 10.134.0.26:8643: connect: connection refused | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-d52cc |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-d52cc-6b7fc7b8b9 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-d52cc-6b7fc7b8b9 |
SuccessfulCreate |
Created pod: ensemble-graph-d52cc-6b7fc7b8b9-nfzxq | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-d52cc-6b7fc7b8b9-nfzxq |
Started |
Started container ensemble-graph-d52cc | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-d52cc-6b7fc7b8b9-nfzxq |
Created |
Created container: ensemble-graph-d52cc | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-d52cc-6b7fc7b8b9-nfzxq |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:dae09a3aa9a23aca96ea2af0400a27c12957dcc91259106b40cbd871a02ad5c3" already present on machine | |
kserve-ci-e2e-test |
multus |
ensemble-graph-d52cc-6b7fc7b8b9-nfzxq |
AddedInterface |
Add eth0 [10.133.0.37/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-d52cc |
InferenceGraphReady |
InferenceGraph [ensemble-graph-d52cc] is Ready | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c2858-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c2858-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d52cc-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-d52cc-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-2e2a5-848d4d594c-z5css |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-c2858-predictor-69ddfb9f7d-bfxf9 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.36:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-c2858-predictor-d76bb676c-2vf88 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.28:8080: connect: connection refused |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-d52cc |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-d52cc |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d52cc-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-d52cc-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c2858-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c2858-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-c2858 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-c2858 |
InferenceServiceReady |
InferenceService [error-404-isvc-c2858] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-c2858 |
InferenceServiceReady |
InferenceService [success-200-isvc-c2858] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-c2858 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-c2858 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-c2858-54876bd879 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-c2858 |
InferenceGraphReady |
InferenceGraph [sequence-graph-c2858] is Ready | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-c2858-54876bd879-lh79j |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "sequence-graph-c2858-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-c2858-54876bd879 |
SuccessfulCreate |
Created pod: sequence-graph-c2858-54876bd879-lh79j | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-c2858-54876bd879-lh79j |
Created |
Created container: sequence-graph-c2858 | |
kserve-ci-e2e-test |
multus |
sequence-graph-c2858-54876bd879-lh79j |
AddedInterface |
Add eth0 [10.133.0.38/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-c2858-54876bd879-lh79j |
Started |
Started container sequence-graph-c2858 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-c2858-54876bd879-lh79j |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:dae09a3aa9a23aca96ea2af0400a27c12957dcc91259106b40cbd871a02ad5c3" already present on machine | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c2858-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-c2858 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c2858-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-c2858 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c2858-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c2858-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-c2858 |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-c2858 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-2fdf6 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-2fdf6": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-2fdf6 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-2fdf6": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-2fdf6 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-2fdf6": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-2fdf6": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d52cc-predictor-5cb644f657-6t22c |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-2fdf6 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-2fdf6": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-2fdf6": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d52cc-predictor-5cb644f657-6t22c |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-2fdf6-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-2fdf6-predictor-74959c8f6c from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-2fdf6-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-2fdf6-predictor-fff5f778b from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-2fdf6-predictor-74959c8f6c |
SuccessfulCreate |
Created pod: success-200-isvc-2fdf6-predictor-74959c8f6c-t8vgn | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d52cc-predictor-65bf9c9c88-b7k7x |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-d52cc-6b7fc7b8b9-nfzxq |
Killing |
Stopping container ensemble-graph-d52cc | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-d52cc-predictor-65bf9c9c88-b7k7x |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-2fdf6-predictor-fff5f778b |
SuccessfulCreate |
Created pod: error-404-isvc-2fdf6-predictor-fff5f778b-5tld6 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2fdf6-predictor-fff5f778b-5tld6 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2fdf6-predictor-fff5f778b-5tld6 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2fdf6-predictor-fff5f778b-5tld6 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2fdf6-predictor-fff5f778b-5tld6 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
success-200-isvc-2fdf6-predictor-74959c8f6c-t8vgn |
AddedInterface |
Add eth0 [10.134.0.29/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2fdf6-predictor-74959c8f6c-t8vgn |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2fdf6-predictor-74959c8f6c-t8vgn |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2fdf6-predictor-74959c8f6c-t8vgn |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2fdf6-predictor-74959c8f6c-t8vgn |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-2fdf6-predictor-fff5f778b-5tld6 |
AddedInterface |
Add eth0 [10.133.0.39/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2fdf6-predictor-74959c8f6c-t8vgn |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2fdf6-predictor-fff5f778b-5tld6 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2fdf6-predictor-74959c8f6c-t8vgn |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1955" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2fdf6-predictor-fff5f778b-5tld6 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1955" already present on machine | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-d52cc-predictor-5cb644f657-6t22c |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.27:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-d52cc-predictor-5cb644f657-6t22c |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.27:8643/healthz": dial tcp 10.134.0.27:8643: connect: connection refused | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2fdf6-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2fdf6-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-d52cc-6b7fc7b8b9-nfzxq |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2fdf6-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2fdf6-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-2fdf6-predictor-fff5f778b-5tld6 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.39:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-2fdf6-predictor-74959c8f6c-t8vgn |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.29:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-2fdf6 |
InferenceServiceReady |
InferenceService [error-404-isvc-2fdf6] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-2fdf6 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-2fdf6 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-2fdf6 |
InferenceServiceReady |
InferenceService [success-200-isvc-2fdf6] is Ready | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c2858-predictor-d76bb676c-2vf88 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c2858-predictor-d76bb676c-2vf88 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-f0d22-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-f0d22-predictor-69c49986b7 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c2858-predictor-69ddfb9f7d-bfxf9 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c2858-predictor-69ddfb9f7d-bfxf9 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-c2858-54876bd879-lh79j |
Killing |
Stopping container sequence-graph-c2858 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f0d22-predictor-69c49986b7-kp9jw |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-f0d22-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0d22-predictor-7667c7cb86-cqmgb |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-f0d22-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-f0d22-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-f0d22-predictor-7667c7cb86 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-f0d22-predictor-7667c7cb86 |
SuccessfulCreate |
Created pod: error-404-isvc-f0d22-predictor-7667c7cb86-cqmgb | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-f0d22-predictor-69c49986b7 |
SuccessfulCreate |
Created pod: success-200-isvc-f0d22-predictor-69c49986b7-kp9jw | |
kserve-ci-e2e-test |
multus |
success-200-isvc-f0d22-predictor-69c49986b7-kp9jw |
AddedInterface |
Add eth0 [10.134.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f0d22-predictor-69c49986b7-kp9jw |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0d22-predictor-7667c7cb86-cqmgb |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0d22-predictor-7667c7cb86-cqmgb |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0d22-predictor-7667c7cb86-cqmgb |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f0d22-predictor-69c49986b7-kp9jw |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1955" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0d22-predictor-7667c7cb86-cqmgb |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0d22-predictor-7667c7cb86-cqmgb |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f0d22-predictor-69c49986b7-kp9jw |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0d22-predictor-7667c7cb86-cqmgb |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1955" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f0d22-predictor-69c49986b7-kp9jw |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f0d22-predictor-69c49986b7-kp9jw |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f0d22-predictor-69c49986b7-kp9jw |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-f0d22-predictor-7667c7cb86-cqmgb |
AddedInterface |
Add eth0 [10.133.0.40/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c2858-predictor-69ddfb9f7d-bfxf9 |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.36:8643/healthz": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c2858-predictor-69ddfb9f7d-bfxf9 |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.36:8080: i/o timeout | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c2858-predictor-d76bb676c-2vf88 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.28:8643/healthz": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c2858-predictor-d76bb676c-2vf88 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.28:8080: i/o timeout | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-2fdf6 |
InferenceGraphReady |
InferenceGraph [splitter-graph-2fdf6] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-2fdf6-7487d76957 |
SuccessfulCreate |
Created pod: splitter-graph-2fdf6-7487d76957-m5hkw | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2fdf6-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2fdf6-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-2fdf6 |
ScalingReplicaSet |
Scaled up replica set splitter-graph-2fdf6-7487d76957 from 0 to 1 | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2fdf6-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2fdf6-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-2fdf6 |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "splitter-graph-2fdf6": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-2fdf6 |
UpdateFailed |
Failed to update status for InferenceGraph "splitter-graph-2fdf6": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "splitter-graph-2fdf6": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-2fdf6-7487d76957-m5hkw |
Started |
Started container splitter-graph-2fdf6 | |
kserve-ci-e2e-test |
multus |
splitter-graph-2fdf6-7487d76957-m5hkw |
AddedInterface |
Add eth0 [10.133.0.41/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-2fdf6-7487d76957-m5hkw |
Created |
Created container: splitter-graph-2fdf6 | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-2fdf6-7487d76957-m5hkw |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:dae09a3aa9a23aca96ea2af0400a27c12957dcc91259106b40cbd871a02ad5c3" already present on machine | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f0d22-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f0d22-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-2fdf6-7487d76957-m5hkw |
Killing |
Stopping container splitter-graph-2fdf6 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-2756e-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-2756e-predictor-58d849df88 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2fdf6-predictor-fff5f778b-5tld6 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2756e-predictor-58d849df88-k4hdz |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-2756e-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-2756e-predictor-5c99844f45 |
SuccessfulCreate |
Created pod: success-200-isvc-2756e-predictor-5c99844f45-rhfvz | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-2756e-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-2756e-predictor-5c99844f45 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2fdf6-predictor-fff5f778b-5tld6 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2fdf6-predictor-74959c8f6c-t8vgn |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-2756e-predictor-58d849df88 |
SuccessfulCreate |
Created pod: error-404-isvc-2756e-predictor-58d849df88-k4hdz | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2756e-predictor-5c99844f45-rhfvz |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2756e-predictor-5c99844f45-rhfvz |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2756e-predictor-5c99844f45-rhfvz |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2756e-predictor-5c99844f45-rhfvz |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2fdf6-predictor-74959c8f6c-t8vgn |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2756e-predictor-5c99844f45-rhfvz |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2756e-predictor-5c99844f45-rhfvz |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-1955" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-2756e-predictor-5c99844f45-rhfvz |
AddedInterface |
Add eth0 [10.134.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2fdf6-predictor-fff5f778b-5tld6 |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.39:8643/healthz": dial tcp 10.133.0.39:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2756e-predictor-58d849df88-k4hdz |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-1955" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2756e-predictor-58d849df88-k4hdz |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2756e-predictor-58d849df88-k4hdz |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2756e-predictor-58d849df88-k4hdz |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2756e-predictor-58d849df88-k4hdz |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2756e-predictor-58d849df88-k4hdz |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-2756e-predictor-58d849df88-k4hdz |
AddedInterface |
Add eth0 [10.133.0.42/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2fdf6-predictor-74959c8f6c-t8vgn |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.29:8643/healthz": dial tcp 10.134.0.29:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-c2858-54876bd879-lh79j |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f0d22-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f0d22-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0d22-predictor-7667c7cb86-cqmgb |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.40:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-f0d22-predictor-69c49986b7-kp9jw |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.30:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-2fdf6-7487d76957-m5hkw |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-f0d22 |
InferenceServiceReady |
InferenceService [success-200-isvc-f0d22] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f0d22 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-f0d22 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-f0d22 |
InferenceServiceReady |
InferenceService [error-404-isvc-f0d22] is Ready | |
| (x6) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-2756e-predictor-5c99844f45-rhfvz |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.31:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-2756e-predictor-58d849df88-k4hdz |
Unhealthy |
Readiness probe failed: dial tcp 10.133.0.42:8080: connect: connection refused |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-f0d22 |
InferenceGraphReady |
InferenceGraph [switch-graph-f0d22] is Ready | |
kserve-ci-e2e-test |
kubelet |
switch-graph-f0d22-6c68dfbc6f-dmg57 |
Started |
Started container switch-graph-f0d22 | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-f0d22-6c68dfbc6f |
SuccessfulCreate |
Created pod: switch-graph-f0d22-6c68dfbc6f-dmg57 | |
kserve-ci-e2e-test |
multus |
switch-graph-f0d22-6c68dfbc6f-dmg57 |
AddedInterface |
Add eth0 [10.133.0.43/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-f0d22-6c68dfbc6f-dmg57 |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:dae09a3aa9a23aca96ea2af0400a27c12957dcc91259106b40cbd871a02ad5c3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
switch-graph-f0d22-6c68dfbc6f-dmg57 |
Created |
Created container: switch-graph-f0d22 | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-f0d22 |
ScalingReplicaSet |
Scaled up replica set switch-graph-f0d22-6c68dfbc6f from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-2756e |
InferenceServiceReady |
InferenceService [success-200-isvc-2756e] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-2756e |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-2756e |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-2756e |
InferenceServiceReady |
InferenceService [error-404-isvc-2756e] is Ready | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f0d22-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f0d22-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-f0d22-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-f0d22-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
kubelet |
splitter-graph-2756e-6c865d8977-c2ssw |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "splitter-graph-2756e-serving-cert" not found | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-2756e |
InferenceGraphReady |
InferenceGraph [splitter-graph-2756e] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-2756e |
ScalingReplicaSet |
Scaled up replica set splitter-graph-2756e-6c865d8977 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-2756e-6c865d8977 |
SuccessfulCreate |
Created pod: splitter-graph-2756e-6c865d8977-c2ssw | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-2756e-6c865d8977-c2ssw |
Started |
Started container splitter-graph-2756e | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-2756e-6c865d8977-c2ssw |
Created |
Created container: splitter-graph-2756e | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-2756e-6c865d8977-c2ssw |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:dae09a3aa9a23aca96ea2af0400a27c12957dcc91259106b40cbd871a02ad5c3" already present on machine | |
kserve-ci-e2e-test |
multus |
splitter-graph-2756e-6c865d8977-c2ssw |
AddedInterface |
Add eth0 [10.133.0.44/23] from ovn-kubernetes | |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2756e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2756e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-2756e |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2756e-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-2756e |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2756e-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-f0d22 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-f0d22 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-2756e |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-2756e |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2756e-predictor-5c99844f45-rhfvz |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2756e-predictor-58d849df88-k4hdz |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-2756e-6c865d8977-c2ssw |
Killing |
Stopping container splitter-graph-2756e | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2756e-predictor-5c99844f45-rhfvz |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2756e-predictor-58d849df88-k4hdz |
Killing |
Stopping container kube-rbac-proxy | |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-2756e-6c865d8977-c2ssw |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
switch-graph-f0d22-6c68dfbc6f-dmg57 |
Killing |
Stopping container switch-graph-f0d22 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f0d22-predictor-69c49986b7-kp9jw |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0d22-predictor-7667c7cb86-cqmgb |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-f0d22-predictor-7667c7cb86-cqmgb |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-f0d22-predictor-69c49986b7-kp9jw |
Killing |
Stopping container kserve-container | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-f0d22-6c68dfbc6f-dmg57 |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |