| Time | Namespace | Component | RelatedObject | Reason | Message |
|---|---|---|---|---|---|
kserve-ci-e2e-test |
success-200-isvc-8cb8c-predictor-765d78496f-kjmd6 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6 to ip-10-0-140-180.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br to ip-10-0-137-48.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6 to ip-10-0-137-48.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2 to ip-10-0-140-180.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm to ip-10-0-140-180.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8 to ip-10-0-137-48.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-c092f-predictor-64b7d65596-7k47n |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n to ip-10-0-137-48.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-ef587-predictor-df4f8586-q6bxt |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt to ip-10-0-137-48.ec2.internal | ||
kserve-ci-e2e-test |
model-chainer-7bcc66cdd8-xhw5g |
Scheduled |
Successfully assigned kserve-ci-e2e-test/model-chainer-7bcc66cdd8-xhw5g to ip-10-0-136-235.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z to ip-10-0-140-180.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-2bced-68fbf74755-bnbzg |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-2bced-68fbf74755-bnbzg to ip-10-0-136-235.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-ef587-6696c94f88-flwcc |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-ef587-6696c94f88-flwcc to ip-10-0-136-235.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-c7ffd-6f479c9f46-cq62h |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-c7ffd-6f479c9f46-cq62h to ip-10-0-136-235.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-c092f-584d475545-7k5nt |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-c092f-584d475545-7k5nt to ip-10-0-136-235.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q to ip-10-0-140-180.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-2bced-predictor-6885566469-55wjx |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx to ip-10-0-140-180.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-8bda6-546cf494dd-ndzbv |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-8bda6-546cf494dd-ndzbv to ip-10-0-136-235.ec2.internal | ||
kserve-ci-e2e-test |
splitter-graph-854fa-7c7884b54-vxpwn |
Scheduled |
Successfully assigned kserve-ci-e2e-test/splitter-graph-854fa-7c7884b54-vxpwn to ip-10-0-136-235.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-e7889-6c74498bf6-9ntcl |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-e7889-6c74498bf6-9ntcl to ip-10-0-136-235.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf to ip-10-0-137-48.ec2.internal | ||
kserve-ci-e2e-test |
ensemble-graph-4d797-7499bc55d8-mpnxg |
Scheduled |
Successfully assigned kserve-ci-e2e-test/ensemble-graph-4d797-7499bc55d8-mpnxg to ip-10-0-136-235.ec2.internal | ||
kserve-ci-e2e-test |
sequence-graph-8cb8c-86db8c7d86-fnnss |
Scheduled |
Successfully assigned kserve-ci-e2e-test/sequence-graph-8cb8c-86db8c7d86-fnnss to ip-10-0-136-235.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt to ip-10-0-137-48.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-2bced-predictor-66fb58dcfb-jlz89 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89 to ip-10-0-137-48.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-854fa-predictor-d9d5c496f-w5djj |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj to ip-10-0-140-180.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-8bda6-predictor-67c7c6c79-bg622 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622 to ip-10-0-140-180.ec2.internal | ||
kserve-ci-e2e-test |
isvc-xgboost-graph-predictor-555fc69d56-xtp66 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66 to ip-10-0-140-180.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-8cb8c-predictor-bdf644456-v5zh5 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5 to ip-10-0-137-48.ec2.internal | ||
kserve-ci-e2e-test |
error-404-isvc-854fa-predictor-5f46c54d88-kcp7l |
Scheduled |
Successfully assigned kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l to ip-10-0-137-48.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-ef587-predictor-6d4f85455-lpmjf |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf to ip-10-0-140-180.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px to ip-10-0-140-180.ec2.internal | ||
kserve-ci-e2e-test |
isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn |
Scheduled |
Successfully assigned kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn to ip-10-0-140-180.ec2.internal | ||
kserve-ci-e2e-test |
success-200-isvc-e7889-predictor-785b95f47f-nvfq8 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8 to ip-10-0-140-180.ec2.internal | ||
kserve-ci-e2e-test |
switch-graph-5b4a7-659766d875-5zr7x |
Scheduled |
Successfully assigned kserve-ci-e2e-test/switch-graph-5b4a7-659766d875-5zr7x to ip-10-0-136-235.ec2.internal | ||
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-1-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-1-predictor-66bf4f5cc4 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xtp66 |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn |
AddedInterface |
Add eth0 [10.134.0.29/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ef587-predictor-df4f8586-q6bxt |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-ef587-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
multus |
isvc-xgboost-graph-predictor-555fc69d56-xtp66 |
AddedInterface |
Add eth0 [10.134.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-ef587-predictor-df4f8586 |
SuccessfulCreate |
Created pod: error-404-isvc-ef587-predictor-df4f8586-q6bxt | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-1-predictor-66bf4f5cc4 |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn | |
kserve-ci-e2e-test |
deployment-controller |
isvc-xgboost-graph-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-xgboost-graph-predictor-555fc69d56 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:2cbd0ba61f740962b2bad1a0e55e41574489d7afdb86fa314f3953b8c8af773e" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-ef587-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-ef587-predictor-df4f8586 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-sklearn-graph-2-predictor-b7ccc5f99 |
SuccessfulCreate |
Created pod: isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q | |
kserve-ci-e2e-test |
replicaset-controller |
isvc-xgboost-graph-predictor-555fc69d56 |
SuccessfulCreate |
Created pod: isvc-xgboost-graph-predictor-555fc69d56-xtp66 | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xtp66 |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-ef587-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-ef587-predictor-6d4f85455 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-ef587-predictor-6d4f85455 |
SuccessfulCreate |
Created pod: success-200-isvc-ef587-predictor-6d4f85455-lpmjf | |
kserve-ci-e2e-test |
deployment-controller |
isvc-sklearn-graph-2-predictor |
ScalingReplicaSet |
Scaled up replica set isvc-sklearn-graph-2-predictor-b7ccc5f99 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xtp66 |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:2cbd0ba61f740962b2bad1a0e55e41574489d7afdb86fa314f3953b8c8af773e" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ef587-predictor-6d4f85455-lpmjf |
Pulling |
Pulling image "quay.io/opendatahub/success-200-isvc:odh-pr-2013" | |
kserve-ci-e2e-test |
multus |
success-200-isvc-ef587-predictor-6d4f85455-lpmjf |
AddedInterface |
Add eth0 [10.134.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:2cbd0ba61f740962b2bad1a0e55e41574489d7afdb86fa314f3953b8c8af773e" already present on machine | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
multus |
error-404-isvc-ef587-predictor-df4f8586-q6bxt |
AddedInterface |
Add eth0 [10.132.0.22/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ef587-predictor-df4f8586-q6bxt |
Pulling |
Pulling image "quay.io/opendatahub/error-404-isvc:odh-pr-2013" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
multus |
isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q |
AddedInterface |
Add eth0 [10.134.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-2013" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q |
Pulling |
Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-2013" | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xtp66 |
Pulling |
Pulling image "kserve/xgbserver:latest" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ef587-predictor-df4f8586-q6bxt |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ef587-predictor-df4f8586-q6bxt |
Pulled |
Successfully pulled image "quay.io/opendatahub/error-404-isvc:odh-pr-2013" in 11.974s (11.974s including waiting). Image size: 1386396243 bytes. | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ef587-predictor-df4f8586-q6bxt |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ef587-predictor-df4f8586-q6bxt |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ef587-predictor-df4f8586-q6bxt |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ef587-predictor-df4f8586-q6bxt |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ef587-predictor-df4f8586-q6bxt |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.202s (2.202s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ef587-predictor-6d4f85455-lpmjf |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-2013" in 13.308s (13.308s including waiting). Image size: 1607124890 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q |
Pulled |
Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-2013" in 12.065s (12.065s including waiting). Image size: 1607124890 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ef587-predictor-6d4f85455-lpmjf |
Pulled |
Successfully pulled image "quay.io/opendatahub/success-200-isvc:odh-pr-2013" in 17.61s (17.61s including waiting). Image size: 1384134229 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ef587-predictor-6d4f85455-lpmjf |
Pulling |
Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ef587-predictor-6d4f85455-lpmjf |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 4.693s (4.693s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ef587-predictor-6d4f85455-lpmjf |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 4.753s (4.753s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 4.458s (4.458s including waiting). Image size: 211946088 bytes. | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ef587-predictor-6d4f85455-lpmjf |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ef587-predictor-6d4f85455-lpmjf |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xtp66 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xtp66 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xtp66 |
Pulled |
Successfully pulled image "kserve/xgbserver:latest" in 32.399s (32.399s including waiting). Image size: 1436472579 bytes. | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xtp66 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xtp66 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xtp66 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-ef587-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-ef587-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-ef587-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-ef587-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x8) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-ef587-predictor-6d4f85455-lpmjf |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.30:8080: connect: connection refused |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-ef587 |
InferenceServiceReady |
InferenceService [error-404-isvc-ef587] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-ef587 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-ef587 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-ef587 |
InferenceServiceReady |
InferenceService [success-200-isvc-ef587] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-ef587-6696c94f88 |
SuccessfulCreate |
Created pod: switch-graph-ef587-6696c94f88-flwcc | |
| (x10) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.29:8080: connect: connection refused |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-ef587 |
ScalingReplicaSet |
Scaled up replica set switch-graph-ef587-6696c94f88 from 0 to 1 | |
| (x10) | kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.32:8080: connect: connection refused |
kserve-ci-e2e-test |
multus |
switch-graph-ef587-6696c94f88-flwcc |
AddedInterface |
Add eth0 [10.133.0.23/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-ef587-6696c94f88-flwcc |
Pulling |
Pulling image "quay.io/opendatahub/kserve-router@sha256:9ee58bd289ecb7aa4010c311b867cae9a40a42edc53d096a1f0354150db5c48e" | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-ef587 |
InferenceGraphReady |
InferenceGraph [switch-graph-ef587] is Ready | |
kserve-ci-e2e-test |
kubelet |
switch-graph-ef587-6696c94f88-flwcc |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-router@sha256:9ee58bd289ecb7aa4010c311b867cae9a40a42edc53d096a1f0354150db5c48e" in 2.371s (2.371s including waiting). Image size: 211751521 bytes. | |
kserve-ci-e2e-test |
kubelet |
switch-graph-ef587-6696c94f88-flwcc |
Created |
Created container: switch-graph-ef587 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-ef587-6696c94f88-flwcc |
Started |
Started container switch-graph-ef587 | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-ef587-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-ef587-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-ef587-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-ef587-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-1] is Ready | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-2 |
InferenceServiceReady |
InferenceService [isvc-sklearn-graph-2] is Ready | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-sklearn-graph-1 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
kubelet |
switch-graph-ef587-6696c94f88-flwcc |
Killing |
Stopping container switch-graph-ef587 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-5b4a7-predictor-7d4878b6fd |
SuccessfulCreate |
Created pod: error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8 | |
kserve-ci-e2e-test |
multus |
success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm |
AddedInterface |
Add eth0 [10.134.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-2013" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-5b4a7-predictor-6fd6897c9f |
SuccessfulCreate |
Created pod: success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-5b4a7-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-5b4a7-predictor-6fd6897c9f from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-5b4a7 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-5b4a7": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-5b4a7": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-5b4a7 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-5b4a7": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-5b4a7 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-5b4a7": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-5b4a7 |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-5b4a7": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-5b4a7": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-5b4a7-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-5b4a7-predictor-7d4878b6fd from 0 to 1 | |
| (x8) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-ef587-predictor-df4f8586-q6bxt |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.22:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ef587-predictor-df4f8586-q6bxt |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ef587-predictor-df4f8586-q6bxt |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-ef587-predictor-df4f8586-q6bxt |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.22:8643/healthz": dial tcp 10.132.0.22:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-5b4a7-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ef587-predictor-6d4f85455-lpmjf |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-ef587-predictor-6d4f85455-lpmjf |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8 |
AddedInterface |
Add eth0 [10.132.0.23/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-2013" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8 |
Created |
Created container: kube-rbac-proxy | |
| (x9) | kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
isvc-xgboost-graph |
InferenceServiceReady |
InferenceService [isvc-xgboost-graph] is Ready | |
kserve-ci-e2e-test |
InferenceGraphController |
model-chainer |
InferenceGraphReady |
InferenceGraph [model-chainer] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
model-chainer-7bcc66cdd8 |
SuccessfulCreate |
Created pod: model-chainer-7bcc66cdd8-xhw5g | |
kserve-ci-e2e-test |
deployment-controller |
model-chainer |
ScalingReplicaSet |
Scaled up replica set model-chainer-7bcc66cdd8 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
model-chainer-7bcc66cdd8-xhw5g |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:9ee58bd289ecb7aa4010c311b867cae9a40a42edc53d096a1f0354150db5c48e" already present on machine | |
kserve-ci-e2e-test |
multus |
model-chainer-7bcc66cdd8-xhw5g |
AddedInterface |
Add eth0 [10.133.0.24/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
model-chainer-7bcc66cdd8-xhw5g |
Created |
Created container: model-chainer | |
kserve-ci-e2e-test |
kubelet |
model-chainer-7bcc66cdd8-xhw5g |
Started |
Started container model-chainer | |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-1-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-xgboost-graph-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
isvc-sklearn-graph-2-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-ef587-6696c94f88-flwcc |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-2bced-predictor-6885566469 |
SuccessfulCreate |
Created pod: success-200-isvc-2bced-predictor-6885566469-55wjx | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-2bced-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-2bced-predictor-66fb58dcfb from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
model-chainer-7bcc66cdd8-xhw5g |
Killing |
Stopping container model-chainer | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-5b4a7-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xtp66 |
Killing |
Stopping container kserve-container | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-5b4a7-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-5b4a7-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-5b4a7-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-2bced-predictor-66fb58dcfb |
SuccessfulCreate |
Created pod: error-404-isvc-2bced-predictor-66fb58dcfb-jlz89 | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-2bced-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-2bced-predictor-6885566469 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xtp66 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2bced-predictor-66fb58dcfb-jlz89 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2bced-predictor-66fb58dcfb-jlz89 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2bced-predictor-6885566469-55wjx |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2bced-predictor-6885566469-55wjx |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2bced-predictor-66fb58dcfb-jlz89 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2bced-predictor-66fb58dcfb-jlz89 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
success-200-isvc-2bced-predictor-6885566469-55wjx |
AddedInterface |
Add eth0 [10.134.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2bced-predictor-6885566469-55wjx |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2bced-predictor-6885566469-55wjx |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-2bced-predictor-66fb58dcfb-jlz89 |
AddedInterface |
Add eth0 [10.132.0.24/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2bced-predictor-66fb58dcfb-jlz89 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2bced-predictor-6885566469-55wjx |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2bced-predictor-66fb58dcfb-jlz89 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-2013" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2bced-predictor-6885566469-55wjx |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-2013" already present on machine | |
| (x10) | kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xtp66 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.31:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.32:8643/healthz": dial tcp 10.134.0.32:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.29:8643/healthz": dial tcp 10.134.0.29:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
isvc-xgboost-graph-predictor-555fc69d56-xtp66 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.31:8643/healthz": dial tcp 10.134.0.31:8643: connect: connection refused | |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.23:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
model-chainer-7bcc66cdd8-xhw5g |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-5b4a7 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-5b4a7 |
InferenceServiceReady |
InferenceService [success-200-isvc-5b4a7] is Ready | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-5b4a7 |
InferenceServiceReady |
InferenceService [error-404-isvc-5b4a7] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-5b4a7 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2bced-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2bced-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2bced-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2bced-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-5b4a7 |
InferenceGraphReady |
InferenceGraph [switch-graph-5b4a7] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-5b4a7 |
ScalingReplicaSet |
Scaled up replica set switch-graph-5b4a7-659766d875 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-5b4a7-659766d875 |
SuccessfulCreate |
Created pod: switch-graph-5b4a7-659766d875-5zr7x | |
kserve-ci-e2e-test |
kubelet |
switch-graph-5b4a7-659766d875-5zr7x |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-5b4a7-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
switch-graph-5b4a7-659766d875-5zr7x |
Started |
Started container switch-graph-5b4a7 | |
kserve-ci-e2e-test |
kubelet |
switch-graph-5b4a7-659766d875-5zr7x |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:9ee58bd289ecb7aa4010c311b867cae9a40a42edc53d096a1f0354150db5c48e" already present on machine | |
kserve-ci-e2e-test |
kubelet |
switch-graph-5b4a7-659766d875-5zr7x |
Created |
Created container: switch-graph-5b4a7 | |
kserve-ci-e2e-test |
multus |
switch-graph-5b4a7-659766d875-5zr7x |
AddedInterface |
Add eth0 [10.133.0.25/23] from ovn-kubernetes | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-5b4a7-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-5b4a7-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-5b4a7-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-5b4a7-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-2bced |
InferenceServiceReady |
InferenceService [success-200-isvc-2bced] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-2bced |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-2bced |
InferenceServiceReady |
InferenceService [error-404-isvc-2bced] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-2bced |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-2bced |
InferenceGraphReady |
InferenceGraph [sequence-graph-2bced] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-2bced-68fbf74755 |
SuccessfulCreate |
Created pod: sequence-graph-2bced-68fbf74755-bnbzg | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-2bced |
ScalingReplicaSet |
Scaled up replica set sequence-graph-2bced-68fbf74755 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
sequence-graph-2bced-68fbf74755-bnbzg |
AddedInterface |
Add eth0 [10.133.0.26/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-2bced-68fbf74755-bnbzg |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:9ee58bd289ecb7aa4010c311b867cae9a40a42edc53d096a1f0354150db5c48e" already present on machine | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-2bced-68fbf74755-bnbzg |
Created |
Created container: sequence-graph-2bced | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-2bced-68fbf74755-bnbzg |
Started |
Started container sequence-graph-2bced | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-5b4a7 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-5b4a7 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2bced-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-2bced-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2bced-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-2bced-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-2bced |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-2bced |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
switch-graph-5b4a7-659766d875-5zr7x |
Killing |
Stopping container switch-graph-5b4a7 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-c092f-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-c092f-predictor-75bf5bdb89 |
SuccessfulCreate |
Created pod: success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-c092f-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-c092f-predictor-75bf5bdb89 from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-c092f-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-c092f-predictor-64b7d65596 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-c092f-predictor-64b7d65596 |
SuccessfulCreate |
Created pod: error-404-isvc-c092f-predictor-64b7d65596-7k47n | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c092f-predictor-64b7d65596-7k47n |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-c092f-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c092f-predictor-64b7d65596-7k47n |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c092f-predictor-64b7d65596-7k47n |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z |
AddedInterface |
Add eth0 [10.134.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-2013" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c092f-predictor-64b7d65596-7k47n |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-c092f-predictor-64b7d65596-7k47n |
AddedInterface |
Add eth0 [10.132.0.25/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c092f-predictor-64b7d65596-7k47n |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-2013" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c092f-predictor-64b7d65596-7k47n |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c092f-predictor-64b7d65596-7k47n |
Started |
Started container kube-rbac-proxy | |
| (x8) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.33:8080: connect: connection refused |
| (x2) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.33:8643/healthz": dial tcp 10.134.0.33:8643: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-5b4a7-659766d875-5zr7x |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2bced-predictor-66fb58dcfb-jlz89 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-e7889-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-e7889-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-e7889-predictor-6f95cf4c98 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-2bced-68fbf74755-bnbzg |
Killing |
Stopping container sequence-graph-2bced | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e7889 |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-e7889": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-e7889-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-e7889-predictor-785b95f47f from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e7889 |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-e7889": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-e7889": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2bced-predictor-66fb58dcfb-jlz89 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2bced-predictor-6885566469-55wjx |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2bced-predictor-6885566469-55wjx |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
success-200-isvc-e7889-predictor-785b95f47f-nvfq8 |
AddedInterface |
Add eth0 [10.134.0.36/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-e7889-predictor-785b95f47f |
SuccessfulCreate |
Created pod: success-200-isvc-e7889-predictor-785b95f47f-nvfq8 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-e7889-predictor-6f95cf4c98 |
SuccessfulCreate |
Created pod: error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-2013" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e7889-predictor-785b95f47f-nvfq8 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e7889-predictor-785b95f47f-nvfq8 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e7889-predictor-785b95f47f-nvfq8 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e7889-predictor-785b95f47f-nvfq8 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e7889-predictor-785b95f47f-nvfq8 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-2013" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6 |
AddedInterface |
Add eth0 [10.132.0.26/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e7889-predictor-785b95f47f-nvfq8 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-2bced-predictor-66fb58dcfb-jlz89 |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.24:8643/healthz": dial tcp 10.132.0.24:8643: connect: connection refused | |
| (x8) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-2bced-predictor-6885566469-55wjx |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.34:8080: connect: connection refused |
| (x8) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-2bced-predictor-66fb58dcfb-jlz89 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.24:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-2bced-predictor-6885566469-55wjx |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.34:8643/healthz": dial tcp 10.134.0.34:8643: connect: connection refused | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c092f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c092f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e7889-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e7889-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.35:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-c092f-predictor-64b7d65596-7k47n |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.25:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-2bced-68fbf74755-bnbzg |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-c092f |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-c092f |
InferenceServiceReady |
InferenceService [success-200-isvc-c092f] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-c092f |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-c092f |
InferenceServiceReady |
InferenceService [error-404-isvc-c092f] is Ready | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c092f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c092f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-c092f |
InferenceGraphReady |
InferenceGraph [ensemble-graph-c092f] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-c092f |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-c092f-584d475545 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-c092f-584d475545 |
SuccessfulCreate |
Created pod: ensemble-graph-c092f-584d475545-7k5nt | |
kserve-ci-e2e-test |
multus |
ensemble-graph-c092f-584d475545-7k5nt |
AddedInterface |
Add eth0 [10.133.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-c092f-584d475545-7k5nt |
Started |
Started container ensemble-graph-c092f | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-c092f-584d475545-7k5nt |
Created |
Created container: ensemble-graph-c092f | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-c092f-584d475545-7k5nt |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:9ee58bd289ecb7aa4010c311b867cae9a40a42edc53d096a1f0354150db5c48e" already present on machine | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c092f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c092f-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c092f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c092f-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-e7889-predictor-785b95f47f-nvfq8 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.36:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.26:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-4d797-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-4d797-predictor-5b98b8c79d from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-4d797-predictor-5b98b8c79d |
SuccessfulCreate |
Created pod: error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-4d797-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-4d797-predictor-86bcd8f8c9 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-c092f-584d475545-7k5nt |
Killing |
Stopping container ensemble-graph-c092f | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c092f-predictor-64b7d65596-7k47n |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c092f-predictor-64b7d65596-7k47n |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-4d797-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-4d797-predictor-86bcd8f8c9 |
SuccessfulCreate |
Created pod: success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-2013" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2 |
AddedInterface |
Add eth0 [10.134.0.37/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf |
AddedInterface |
Add eth0 [10.132.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-2013" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.35:8643/healthz": dial tcp 10.134.0.35:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c092f-predictor-64b7d65596-7k47n |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.25:8643/healthz": dial tcp 10.132.0.25:8643: connect: connection refused | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e7889 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-e7889 |
InferenceServiceReady |
InferenceService [success-200-isvc-e7889] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e7889 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-e7889 |
InferenceServiceReady |
InferenceService [error-404-isvc-e7889] is Ready | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-e7889 |
UpdateFailed |
Failed to update status for InferenceGraph "sequence-graph-e7889": Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "sequence-graph-e7889": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-e7889 |
InferenceGraphReady |
InferenceGraph [sequence-graph-e7889] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-e7889 |
ScalingReplicaSet |
Scaled up replica set sequence-graph-e7889-6c74498bf6 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-e7889-6c74498bf6 |
SuccessfulCreate |
Created pod: sequence-graph-e7889-6c74498bf6-9ntcl | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-e7889 |
InternalError |
fails to update InferenceGraph status: Operation cannot be fulfilled on inferencegraphs.serving.kserve.io "sequence-graph-e7889": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-e7889-6c74498bf6-9ntcl |
Started |
Started container sequence-graph-e7889 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-e7889-6c74498bf6-9ntcl |
Created |
Created container: sequence-graph-e7889 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-e7889-6c74498bf6-9ntcl |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:9ee58bd289ecb7aa4010c311b867cae9a40a42edc53d096a1f0354150db5c48e" already present on machine | |
kserve-ci-e2e-test |
multus |
sequence-graph-e7889-6c74498bf6-9ntcl |
AddedInterface |
Add eth0 [10.133.0.28/23] from ovn-kubernetes | |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e7889-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e7889-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-e7889-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-e7889-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-c092f-584d475545-7k5nt |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4d797-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4d797-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4d797-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4d797-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-8cb8c-predictor-765d78496f |
SuccessfulCreate |
Created pod: success-200-isvc-8cb8c-predictor-765d78496f-kjmd6 | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-e7889-6c74498bf6-9ntcl |
Killing |
Stopping container sequence-graph-e7889 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-8cb8c-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-8cb8c-predictor-bdf644456 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-8cb8c-predictor-bdf644456 |
SuccessfulCreate |
Created pod: error-404-isvc-8cb8c-predictor-bdf644456-v5zh5 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-8cb8c |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-8cb8c": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-8cb8c-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-8cb8c-predictor-765d78496f from 0 to 1 | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-8cb8c |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-8cb8c": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-8cb8c": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-8cb8c |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-8cb8c": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-8cb8c |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-8cb8c": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-8cb8c": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e7889-predictor-785b95f47f-nvfq8 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e7889-predictor-785b95f47f-nvfq8 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8cb8c-predictor-765d78496f-kjmd6 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-8cb8c-predictor-bdf644456-v5zh5 |
AddedInterface |
Add eth0 [10.132.0.28/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
success-200-isvc-8cb8c-predictor-765d78496f-kjmd6 |
AddedInterface |
Add eth0 [10.134.0.38/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8cb8c-predictor-765d78496f-kjmd6 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8cb8c-predictor-bdf644456-v5zh5 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8cb8c-predictor-765d78496f-kjmd6 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-2013" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8cb8c-predictor-765d78496f-kjmd6 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8cb8c-predictor-bdf644456-v5zh5 |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-2013" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8cb8c-predictor-bdf644456-v5zh5 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8cb8c-predictor-765d78496f-kjmd6 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8cb8c-predictor-bdf644456-v5zh5 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8cb8c-predictor-bdf644456-v5zh5 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8cb8c-predictor-bdf644456-v5zh5 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8cb8c-predictor-765d78496f-kjmd6 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-e7889-predictor-785b95f47f-nvfq8 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.36:8643/healthz": dial tcp 10.134.0.36:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6 |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.26:8643/healthz": dial tcp 10.132.0.26:8643: connect: connection refused | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8cb8c-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8cb8c-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.37:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-e7889-6c74498bf6-9ntcl |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-4d797 |
InferenceServiceReady |
InferenceService [success-200-isvc-4d797] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-4d797 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-4d797 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-4d797 |
InferenceServiceReady |
InferenceService [error-404-isvc-4d797] is Ready | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8cb8c-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8cb8c-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
InferenceGraphController |
ensemble-graph-4d797 |
InferenceGraphReady |
InferenceGraph [ensemble-graph-4d797] is Ready | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-4d797-7499bc55d8-mpnxg |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "ensemble-graph-4d797-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
ensemble-graph-4d797-7499bc55d8 |
SuccessfulCreate |
Created pod: ensemble-graph-4d797-7499bc55d8-mpnxg | |
kserve-ci-e2e-test |
deployment-controller |
ensemble-graph-4d797 |
ScalingReplicaSet |
Scaled up replica set ensemble-graph-4d797-7499bc55d8 from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-4d797-7499bc55d8-mpnxg |
Started |
Started container ensemble-graph-4d797 | |
kserve-ci-e2e-test |
multus |
ensemble-graph-4d797-7499bc55d8-mpnxg |
AddedInterface |
Add eth0 [10.133.0.29/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-4d797-7499bc55d8-mpnxg |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:9ee58bd289ecb7aa4010c311b867cae9a40a42edc53d096a1f0354150db5c48e" already present on machine | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-4d797-7499bc55d8-mpnxg |
Created |
Created container: ensemble-graph-4d797 | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-8cb8c |
InferenceServiceReady |
InferenceService [success-200-isvc-8cb8c] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-8cb8c |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4d797-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4d797-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-8cb8c |
InferenceServiceReady |
InferenceService [error-404-isvc-8cb8c] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-8cb8c |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-4d797-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-4d797-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
kserve-ci-e2e-test |
deployment-controller |
sequence-graph-8cb8c |
ScalingReplicaSet |
Scaled up replica set sequence-graph-8cb8c-86db8c7d86 from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
sequence-graph-8cb8c |
InferenceGraphReady |
InferenceGraph [sequence-graph-8cb8c] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
sequence-graph-8cb8c-86db8c7d86 |
SuccessfulCreate |
Created pod: sequence-graph-8cb8c-86db8c7d86-fnnss | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-8cb8c-86db8c7d86-fnnss |
Started |
Started container sequence-graph-8cb8c | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-8cb8c-86db8c7d86-fnnss |
Created |
Created container: sequence-graph-8cb8c | |
kserve-ci-e2e-test |
kubelet |
sequence-graph-8cb8c-86db8c7d86-fnnss |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:9ee58bd289ecb7aa4010c311b867cae9a40a42edc53d096a1f0354150db5c48e" already present on machine | |
kserve-ci-e2e-test |
multus |
sequence-graph-8cb8c-86db8c7d86-fnnss |
AddedInterface |
Add eth0 [10.133.0.30/23] from ovn-kubernetes | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8cb8c-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8cb8c-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8cb8c-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8cb8c-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-4d797 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
ensemble-graph-4d797 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-8cb8c |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
sequence-graph-8cb8c |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-854fa-predictor-d9d5c496f |
SuccessfulCreate |
Created pod: success-200-isvc-854fa-predictor-d9d5c496f-w5djj | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-854fa |
UpdateFailed |
Failed to update status for InferenceService "error-404-isvc-854fa": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-854fa": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-854fa-predictor-d9d5c496f-w5djj |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-854fa-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-854fa-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-854fa-predictor-d9d5c496f from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
ensemble-graph-4d797-7499bc55d8-mpnxg |
Killing |
Stopping container ensemble-graph-4d797 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-854fa |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "error-404-isvc-854fa": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-854fa |
UpdateFailed |
Failed to update status for InferenceService "success-200-isvc-854fa": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-854fa": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-854fa |
InternalError |
fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "success-200-isvc-854fa": the object has been modified; please apply your changes to the latest version and try again | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-854fa-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-854fa-predictor-5f46c54d88 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-854fa-predictor-5f46c54d88 |
SuccessfulCreate |
Created pod: error-404-isvc-854fa-predictor-5f46c54d88-kcp7l | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-854fa-predictor-5f46c54d88-kcp7l |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-854fa-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
multus |
success-200-isvc-854fa-predictor-d9d5c496f-w5djj |
AddedInterface |
Add eth0 [10.134.0.39/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-854fa-predictor-d9d5c496f-w5djj |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-854fa-predictor-5f46c54d88-kcp7l |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-2013" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-854fa-predictor-5f46c54d88-kcp7l |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-854fa-predictor-d9d5c496f-w5djj |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-854fa-predictor-5f46c54d88-kcp7l |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-854fa-predictor-d9d5c496f-w5djj |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-854fa-predictor-d9d5c496f-w5djj |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-854fa-predictor-5f46c54d88-kcp7l |
AddedInterface |
Add eth0 [10.132.0.29/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-854fa-predictor-5f46c54d88-kcp7l |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-854fa-predictor-d9d5c496f-w5djj |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-854fa-predictor-d9d5c496f-w5djj |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-2013" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-854fa-predictor-5f46c54d88-kcp7l |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-854fa-predictor-5f46c54d88-kcp7l |
Started |
Started container kube-rbac-proxy | |
| (x8) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.27:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.27:8643/healthz": dial tcp 10.132.0.27:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.37:8643/healthz": context deadline exceeded | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.37:8080: i/o timeout | |
| (x6) | kserve-ci-e2e-test |
kubelet |
ensemble-graph-4d797-7499bc55d8-mpnxg |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-854fa-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-854fa-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
sequence-graph-8cb8c-86db8c7d86-fnnss |
Killing |
Stopping container sequence-graph-8cb8c | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8cb8c-predictor-bdf644456-v5zh5 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-c7ffd-predictor-6597cccd7b |
SuccessfulCreate |
Created pod: error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-c7ffd-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-c7ffd-predictor-7f499f4f6f from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-c7ffd-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-c7ffd-predictor-6597cccd7b from 0 to 1 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8cb8c-predictor-765d78496f-kjmd6 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8cb8c-predictor-765d78496f-kjmd6 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-2013" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px |
AddedInterface |
Add eth0 [10.134.0.40/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-c7ffd-predictor-7f499f4f6f |
SuccessfulCreate |
Created pod: success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8cb8c-predictor-bdf644456-v5zh5 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
multus |
error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br |
AddedInterface |
Add eth0 [10.132.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-2013" already present on machine | |
| (x8) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-8cb8c-predictor-765d78496f-kjmd6 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.38:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8cb8c-predictor-765d78496f-kjmd6 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.38:8643/healthz": dial tcp 10.134.0.38:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8cb8c-predictor-bdf644456-v5zh5 |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.28:8643/healthz": dial tcp 10.132.0.28:8643: connect: connection refused | |
| (x8) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-8cb8c-predictor-bdf644456-v5zh5 |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.28:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-854fa-predictor-5f46c54d88-kcp7l |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.29:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-854fa-predictor-d9d5c496f-w5djj |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.39:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-854fa |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-854fa |
InferenceServiceReady |
InferenceService [error-404-isvc-854fa] is Ready | |
| (x6) | kserve-ci-e2e-test |
kubelet |
sequence-graph-8cb8c-86db8c7d86-fnnss |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-854fa |
InferenceServiceReady |
InferenceService [success-200-isvc-854fa] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-854fa |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c7ffd-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c7ffd-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c7ffd-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c7ffd-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-854fa |
InferenceGraphReady |
InferenceGraph [splitter-graph-854fa] is Ready | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-854fa-7c7884b54-vxpwn |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "splitter-graph-854fa-serving-cert" not found | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-854fa |
ScalingReplicaSet |
Scaled up replica set splitter-graph-854fa-7c7884b54 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-854fa-7c7884b54 |
SuccessfulCreate |
Created pod: splitter-graph-854fa-7c7884b54-vxpwn | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-854fa-7c7884b54-vxpwn |
Created |
Created container: splitter-graph-854fa | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-854fa-7c7884b54-vxpwn |
Started |
Started container splitter-graph-854fa | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-854fa-7c7884b54-vxpwn |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:9ee58bd289ecb7aa4010c311b867cae9a40a42edc53d096a1f0354150db5c48e" already present on machine | |
kserve-ci-e2e-test |
multus |
splitter-graph-854fa-7c7884b54-vxpwn |
AddedInterface |
Add eth0 [10.133.0.31/23] from ovn-kubernetes | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-854fa-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-854fa-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x5) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-854fa-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-854fa-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.30:8080: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.40:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-854fa-predictor-d9d5c496f-w5djj |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-854fa-predictor-5f46c54d88-kcp7l |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-854fa-7c7884b54-vxpwn |
Killing |
Stopping container splitter-graph-854fa | |
kserve-ci-e2e-test |
deployment-controller |
error-404-isvc-8bda6-predictor |
ScalingReplicaSet |
Scaled up replica set error-404-isvc-8bda6-predictor-69655b5cb6 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
error-404-isvc-8bda6-predictor-69655b5cb6 |
SuccessfulCreate |
Created pod: error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt | |
kserve-ci-e2e-test |
deployment-controller |
success-200-isvc-8bda6-predictor |
ScalingReplicaSet |
Scaled up replica set success-200-isvc-8bda6-predictor-67c7c6c79 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
success-200-isvc-8bda6-predictor-67c7c6c79 |
SuccessfulCreate |
Created pod: success-200-isvc-8bda6-predictor-67c7c6c79-bg622 | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-854fa-predictor-d9d5c496f-w5djj |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-854fa-predictor-5f46c54d88-kcp7l |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8bda6-predictor-67c7c6c79-bg622 |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "success-200-isvc-8bda6-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "error-404-isvc-8bda6-predictor-serving-cert" not found | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8bda6-predictor-67c7c6c79-bg622 |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt |
Created |
Created container: kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-854fa-predictor-5f46c54d88-kcp7l |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.29:8643/healthz": dial tcp 10.132.0.29:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8bda6-predictor-67c7c6c79-bg622 |
Started |
Started container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt |
Pulled |
Container image "quay.io/opendatahub/error-404-isvc:odh-pr-2013" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8bda6-predictor-67c7c6c79-bg622 |
Created |
Created container: kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8bda6-predictor-67c7c6c79-bg622 |
Pulled |
Container image "quay.io/opendatahub/success-200-isvc:odh-pr-2013" already present on machine | |
kserve-ci-e2e-test |
multus |
success-200-isvc-8bda6-predictor-67c7c6c79-bg622 |
AddedInterface |
Add eth0 [10.134.0.41/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8bda6-predictor-67c7c6c79-bg622 |
Started |
Started container kube-rbac-proxy | |
kserve-ci-e2e-test |
multus |
error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt |
AddedInterface |
Add eth0 [10.132.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8bda6-predictor-67c7c6c79-bg622 |
Pulled |
Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-854fa-predictor-d9d5c496f-w5djj |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.39:8643/healthz": dial tcp 10.134.0.39:8643: connect: connection refused | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-c7ffd |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-c7ffd |
InferenceServiceReady |
InferenceService [error-404-isvc-c7ffd] is Ready | |
| (x5) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-c7ffd |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-c7ffd |
InferenceServiceReady |
InferenceService [success-200-isvc-c7ffd] is Ready | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8bda6-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8bda6-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API | |
kserve-ci-e2e-test |
kubelet |
switch-graph-c7ffd-6f479c9f46-cq62h |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "switch-graph-c7ffd-serving-cert" not found | |
kserve-ci-e2e-test |
replicaset-controller |
switch-graph-c7ffd-6f479c9f46 |
SuccessfulCreate |
Created pod: switch-graph-c7ffd-6f479c9f46-cq62h | |
kserve-ci-e2e-test |
InferenceGraphController |
switch-graph-c7ffd |
InferenceGraphReady |
InferenceGraph [switch-graph-c7ffd] is Ready | |
kserve-ci-e2e-test |
deployment-controller |
switch-graph-c7ffd |
ScalingReplicaSet |
Scaled up replica set switch-graph-c7ffd-6f479c9f46 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
switch-graph-c7ffd-6f479c9f46-cq62h |
AddedInterface |
Add eth0 [10.133.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
switch-graph-c7ffd-6f479c9f46-cq62h |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:9ee58bd289ecb7aa4010c311b867cae9a40a42edc53d096a1f0354150db5c48e" already present on machine | |
kserve-ci-e2e-test |
kubelet |
switch-graph-c7ffd-6f479c9f46-cq62h |
Created |
Created container: switch-graph-c7ffd | |
kserve-ci-e2e-test |
kubelet |
switch-graph-c7ffd-6f479c9f46-cq62h |
Started |
Started container switch-graph-c7ffd | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c7ffd-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-c7ffd-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c7ffd-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-c7ffd-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-854fa-7c7884b54-vxpwn |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8bda6-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8bda6-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x7) | kserve-ci-e2e-test |
kubelet |
error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.31:8080: connect: connection refused |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-8bda6 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
success-200-isvc-8bda6 |
InferenceServiceReady |
InferenceService [success-200-isvc-8bda6] is Ready | |
| (x6) | kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-8bda6 |
VirtualServiceCRDNotFound |
Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true. |
kserve-ci-e2e-test |
v1beta1Controllers |
error-404-isvc-8bda6 |
InferenceServiceReady |
InferenceService [error-404-isvc-8bda6] is Ready | |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-c7ffd |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x3) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
switch-graph-c7ffd |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
replicaset-controller |
splitter-graph-8bda6-546cf494dd |
SuccessfulCreate |
Created pod: splitter-graph-8bda6-546cf494dd-ndzbv | |
kserve-ci-e2e-test |
deployment-controller |
splitter-graph-8bda6 |
ScalingReplicaSet |
Scaled up replica set splitter-graph-8bda6-546cf494dd from 0 to 1 | |
kserve-ci-e2e-test |
InferenceGraphController |
splitter-graph-8bda6 |
InferenceGraphReady |
InferenceGraph [splitter-graph-8bda6] is Ready | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-8bda6-546cf494dd-ndzbv |
FailedMount |
MountVolume.SetUp failed for volume "proxy-tls" : secret "splitter-graph-8bda6-serving-cert" not found | |
kserve-ci-e2e-test |
multus |
splitter-graph-8bda6-546cf494dd-ndzbv |
AddedInterface |
Add eth0 [10.133.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-8bda6-546cf494dd-ndzbv |
Started |
Started container splitter-graph-8bda6 | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-8bda6-546cf494dd-ndzbv |
Pulled |
Container image "quay.io/opendatahub/kserve-router@sha256:9ee58bd289ecb7aa4010c311b867cae9a40a42edc53d096a1f0354150db5c48e" already present on machine | |
kserve-ci-e2e-test |
kubelet |
splitter-graph-8bda6-546cf494dd-ndzbv |
Created |
Created container: splitter-graph-8bda6 | |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8bda6-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
success-200-isvc-8bda6-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8bda6-predictor |
FailedGetResourceMetric |
failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x4) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
error-404-isvc-8bda6-predictor |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-8bda6 |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | kserve-ci-e2e-test |
horizontal-pod-autoscaler |
splitter-graph-8bda6 |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kserve-ci-e2e-test |
kubelet |
splitter-graph-8bda6-546cf494dd-ndzbv |
Killing |
Stopping container splitter-graph-8bda6 | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8bda6-predictor-67c7c6c79-bg622 |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8bda6-predictor-67c7c6c79-bg622 |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-8bda6-predictor-67c7c6c79-bg622 |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.41:8643/healthz": dial tcp 10.134.0.41:8643: connect: connection refused | |
| (x8) | kserve-ci-e2e-test |
kubelet |
success-200-isvc-8bda6-predictor-67c7c6c79-bg622 |
Unhealthy |
Readiness probe failed: dial tcp 10.134.0.41:8080: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.31:8643/healthz": context deadline exceeded | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt |
Unhealthy |
Readiness probe failed: dial tcp 10.132.0.31:8080: i/o timeout | |
| (x6) | kserve-ci-e2e-test |
kubelet |
splitter-graph-8bda6-546cf494dd-ndzbv |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
switch-graph-c7ffd-6f479c9f46-cq62h |
Killing |
Stopping container switch-graph-c7ffd | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px |
Killing |
Stopping container kube-rbac-proxy | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px |
Killing |
Stopping container kserve-container | |
kserve-ci-e2e-test |
kubelet |
error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.30:8643/healthz": dial tcp 10.132.0.30:8643: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px |
Unhealthy |
Readiness probe failed: Get "https://10.134.0.40:8643/healthz": dial tcp 10.134.0.40:8643: connect: connection refused | |
| (x6) | kserve-ci-e2e-test |
kubelet |
switch-graph-c7ffd-6f479c9f46-cq62h |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 |