Time Namespace Component RelatedObject Reason Message

kserve-ci-e2e-test

message-dumper-predictor-6575bcfdfd-v85ks

Scheduled

 Successfully assigned kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks to ip-10-0-134-48.ec2.internal

kserve-ci-e2e-test

isvc-autogluon-v2-predictor-67b849d456-xg4r6

Scheduled

 Successfully assigned kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6 to ip-10-0-134-48.ec2.internal

kserve-ci-e2e-test

isvc-sklearn-batcher-predictor-78b5fd7767-47skv

Scheduled

 Successfully assigned kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv to ip-10-0-134-48.ec2.internal

kserve-ci-e2e-test

isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2

Scheduled

 Successfully assigned kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2 to ip-10-0-134-48.ec2.internal

kserve-ci-e2e-test

isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb

Scheduled

 Successfully assigned kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb to ip-10-0-134-48.ec2.internal

kserve-ci-e2e-test

isvc-logger-predictor-5d9d8fb8b4-c549m

Scheduled

 Successfully assigned kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m to ip-10-0-134-48.ec2.internal

kserve-ci-e2e-test

isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt

Scheduled

 Successfully assigned kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt to ip-10-0-134-48.ec2.internal

kserve-ci-e2e-test

isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn

Scheduled

 Successfully assigned kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn to ip-10-0-134-48.ec2.internal

kserve-ci-e2e-test

isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw

Scheduled

 Successfully assigned kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw to ip-10-0-134-48.ec2.internal

kserve-ci-e2e-test

deployment-controller

isvc-sklearn-batcher-predictor

ScalingReplicaSet

 Scaled up replica set isvc-sklearn-batcher-predictor-78b5fd7767 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

isvc-sklearn-batcher-predictor-78b5fd7767

SuccessfulCreate

 Created pod: isvc-sklearn-batcher-predictor-78b5fd7767-47skv

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-predictor-78b5fd7767-47skv

Started

 Started container storage-initializer

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-predictor-78b5fd7767-47skv

Created

 Created container: storage-initializer

kserve-ci-e2e-test

multus

isvc-sklearn-batcher-predictor-78b5fd7767-47skv

AddedInterface

 Add eth0 [10.134.0.26/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-predictor-78b5fd7767-47skv

Pulled

 Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine
(x2)

kserve-ci-e2e-test

v1beta1Controllers

isvc-sklearn-batcher

UpdateFailed

 Failed to update status for InferenceService "isvc-sklearn-batcher": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "isvc-sklearn-batcher": the object has been modified; please apply your changes to the latest version and try again
(x2)

kserve-ci-e2e-test

v1beta1Controllers

isvc-sklearn-batcher

InternalError

 fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "isvc-sklearn-batcher": the object has been modified; please apply your changes to the latest version and try again

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-predictor-78b5fd7767-47skv

Pulling

 Pulling image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1730"

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-predictor-78b5fd7767-47skv

Created

 Created container: kserve-container

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-predictor-78b5fd7767-47skv

Started

 Started container kserve-container

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-predictor-78b5fd7767-47skv

Pulling

 Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3"

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-predictor-78b5fd7767-47skv

Pulled

 Successfully pulled image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1730" in 14.939s (14.939s including waiting). Image size: 1555978657 bytes.

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-predictor-78b5fd7767-47skv

Started

 Started container kube-rbac-proxy

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-predictor-78b5fd7767-47skv

Pulling

 Pulling image "quay.io/opendatahub/kserve-agent@sha256:4e90f566b342ee88137b4332294c4cef6db04c33dbbb6e9394e58bda8902c06d"

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-predictor-78b5fd7767-47skv

Created

 Created container: kube-rbac-proxy

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-predictor-78b5fd7767-47skv

Pulled

 Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" in 2.116s (2.116s including waiting). Image size: 211946088 bytes.

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-predictor-78b5fd7767-47skv

Pulled

 Successfully pulled image "quay.io/opendatahub/kserve-agent@sha256:4e90f566b342ee88137b4332294c4cef6db04c33dbbb6e9394e58bda8902c06d" in 2.831s (2.831s including waiting). Image size: 239997016 bytes.

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-predictor-78b5fd7767-47skv

Started

 Started container agent

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-predictor-78b5fd7767-47skv

Created

 Created container: agent
(x3)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-sklearn-batcher-predictor

FailedGetResourceMetric

 failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-sklearn-batcher-predictor

FailedComputeMetricsReplicas

 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-sklearn-batcher-predictor

FailedComputeMetricsReplicas

 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-sklearn-batcher-predictor

FailedGetResourceMetric

 failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x11)

kserve-ci-e2e-test

v1beta1Controllers

isvc-sklearn-batcher

VirtualServiceCRDNotFound

 Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true.

kserve-ci-e2e-test

v1beta1Controllers

isvc-sklearn-batcher

InferenceServiceReady

 InferenceService [isvc-sklearn-batcher] is Ready

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-predictor-78b5fd7767-47skv

Killing

 Stopping container kserve-container

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-predictor-78b5fd7767-47skv

Killing

 Stopping container agent

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-predictor-78b5fd7767-47skv

Killing

 Stopping container kube-rbac-proxy

kserve-ci-e2e-test

deployment-controller

isvc-sklearn-batcher-custom-predictor

ScalingReplicaSet

 Scaled up replica set isvc-sklearn-batcher-custom-predictor-5f9bdc45b6 from 0 to 1

kserve-ci-e2e-test

v1beta1Controllers

isvc-sklearn-batcher-custom

UpdateFailed

 Failed to update status for InferenceService "isvc-sklearn-batcher-custom": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "isvc-sklearn-batcher-custom": the object has been modified; please apply your changes to the latest version and try again

kserve-ci-e2e-test

replicaset-controller

isvc-sklearn-batcher-custom-predictor-5f9bdc45b6

SuccessfulCreate

 Created pod: isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2

kserve-ci-e2e-test

multus

isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2

AddedInterface

 Add eth0 [10.134.0.27/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2

Pulled

 Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2

Created

 Created container: storage-initializer

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2

Started

 Started container storage-initializer

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2

Started

 Started container kube-rbac-proxy

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2

Created

 Created container: agent

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2

Started

 Started container kserve-container

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2

Created

 Created container: kserve-container

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2

Pulled

 Container image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1730" already present on machine

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2

Pulled

 Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2

Created

 Created container: kube-rbac-proxy

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2

Started

 Started container agent

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2

Pulled

 Container image "quay.io/opendatahub/kserve-agent@sha256:4e90f566b342ee88137b4332294c4cef6db04c33dbbb6e9394e58bda8902c06d" already present on machine
(x3)

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-predictor-78b5fd7767-47skv

Unhealthy

 Readiness probe failed: Get "https://10.134.0.26:8643/healthz": dial tcp 10.134.0.26:8643: connect: connection refused
(x11)

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-predictor-78b5fd7767-47skv

Unhealthy

 Readiness probe failed: dial tcp 10.134.0.26:8080: connect: connection refused
(x11)

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-predictor-78b5fd7767-47skv

Unhealthy

 Readiness probe failed: HTTP probe failed with statuscode: 503
(x2)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-sklearn-batcher-custom-predictor

FailedComputeMetricsReplicas

 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-sklearn-batcher-custom-predictor

FailedGetResourceMetric

 failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-sklearn-batcher-custom-predictor

FailedComputeMetricsReplicas

 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-sklearn-batcher-custom-predictor

FailedGetResourceMetric

 failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

kserve-ci-e2e-test

v1beta1Controllers

isvc-sklearn-batcher-custom

InferenceServiceReady

 InferenceService [isvc-sklearn-batcher-custom] is Ready
(x11)

kserve-ci-e2e-test

v1beta1Controllers

isvc-sklearn-batcher-custom

VirtualServiceCRDNotFound

 Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true.

kserve-ci-e2e-test

v1beta1Controllers

message-dumper

UpdateFailed

 Failed to update status for InferenceService "message-dumper": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "message-dumper": the object has been modified; please apply your changes to the latest version and try again

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2

Killing

 Stopping container kube-rbac-proxy

kserve-ci-e2e-test

deployment-controller

message-dumper-predictor

ScalingReplicaSet

 Scaled up replica set message-dumper-predictor-6575bcfdfd from 0 to 1

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2

Killing

 Stopping container kserve-container

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2

Killing

 Stopping container agent

kserve-ci-e2e-test

multus

message-dumper-predictor-6575bcfdfd-v85ks

AddedInterface

 Add eth0 [10.134.0.28/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

message-dumper-predictor-6575bcfdfd-v85ks

Pulling

 Pulling image "gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display"

kserve-ci-e2e-test

replicaset-controller

message-dumper-predictor-6575bcfdfd

SuccessfulCreate

 Created pod: message-dumper-predictor-6575bcfdfd-v85ks

kserve-ci-e2e-test

kubelet

message-dumper-predictor-6575bcfdfd-v85ks

Pulled

 Successfully pulled image "gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display" in 1.144s (1.144s including waiting). Image size: 14813193 bytes.

kserve-ci-e2e-test

kubelet

message-dumper-predictor-6575bcfdfd-v85ks

Started

 Started container kserve-container

kserve-ci-e2e-test

kubelet

message-dumper-predictor-6575bcfdfd-v85ks

Pulled

 Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine

kserve-ci-e2e-test

kubelet

message-dumper-predictor-6575bcfdfd-v85ks

Created

 Created container: kserve-container

kserve-ci-e2e-test

kubelet

message-dumper-predictor-6575bcfdfd-v85ks

Created

 Created container: kube-rbac-proxy

kserve-ci-e2e-test

kubelet

message-dumper-predictor-6575bcfdfd-v85ks

Started

 Started container kube-rbac-proxy

kserve-ci-e2e-test

v1beta1Controllers

message-dumper

InferenceServiceReady

 InferenceService [message-dumper] is Ready
(x7)

kserve-ci-e2e-test

v1beta1Controllers

message-dumper

VirtualServiceCRDNotFound

 Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true.
(x3)

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2

Unhealthy

 Readiness probe failed: Get "https://10.134.0.27:8643/healthz": dial tcp 10.134.0.27:8643: connect: connection refused
(x11)

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2

Unhealthy

 Readiness probe failed: HTTP probe failed with statuscode: 503
(x11)

kserve-ci-e2e-test

kubelet

isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2

Unhealthy

 Readiness probe failed: dial tcp 10.134.0.27:5000: connect: connection refused

kserve-ci-e2e-test

deployment-controller

isvc-logger-predictor

ScalingReplicaSet

 Scaled up replica set isvc-logger-predictor-5d9d8fb8b4 from 0 to 1

kserve-ci-e2e-test

kubelet

isvc-logger-predictor-5d9d8fb8b4-c549m

Started

 Started container storage-initializer

kserve-ci-e2e-test

kubelet

isvc-logger-predictor-5d9d8fb8b4-c549m

Created

 Created container: storage-initializer

kserve-ci-e2e-test

kubelet

isvc-logger-predictor-5d9d8fb8b4-c549m

Pulled

 Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

multus

isvc-logger-predictor-5d9d8fb8b4-c549m

AddedInterface

 Add eth0 [10.134.0.29/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

isvc-logger-predictor-5d9d8fb8b4

SuccessfulCreate

 Created pod: isvc-logger-predictor-5d9d8fb8b4-c549m

kserve-ci-e2e-test

v1beta1Controllers

isvc-logger

UpdateFailed

 Failed to update status for InferenceService "isvc-logger": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "isvc-logger": the object has been modified; please apply your changes to the latest version and try again

kserve-ci-e2e-test

kubelet

isvc-logger-predictor-5d9d8fb8b4-c549m

Pulled

 Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine

kserve-ci-e2e-test

kubelet

isvc-logger-predictor-5d9d8fb8b4-c549m

Started

 Started container kube-rbac-proxy

kserve-ci-e2e-test

kubelet

isvc-logger-predictor-5d9d8fb8b4-c549m

Pulled

 Container image "quay.io/opendatahub/kserve-agent@sha256:4e90f566b342ee88137b4332294c4cef6db04c33dbbb6e9394e58bda8902c06d" already present on machine

kserve-ci-e2e-test

kubelet

isvc-logger-predictor-5d9d8fb8b4-c549m

Created

 Created container: kube-rbac-proxy

kserve-ci-e2e-test

kubelet

isvc-logger-predictor-5d9d8fb8b4-c549m

Created

 Created container: kserve-container

kserve-ci-e2e-test

kubelet

isvc-logger-predictor-5d9d8fb8b4-c549m

Pulled

 Container image "quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1730" already present on machine

kserve-ci-e2e-test

kubelet

isvc-logger-predictor-5d9d8fb8b4-c549m

Started

 Started container agent

kserve-ci-e2e-test

kubelet

isvc-logger-predictor-5d9d8fb8b4-c549m

Created

 Created container: agent

kserve-ci-e2e-test

kubelet

isvc-logger-predictor-5d9d8fb8b4-c549m

Started

 Started container kserve-container

kserve-ci-e2e-test

horizontal-pod-autoscaler

message-dumper-predictor

FailedComputeMetricsReplicas

 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

kserve-ci-e2e-test

horizontal-pod-autoscaler

message-dumper-predictor

FailedGetResourceMetric

 failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

kserve-ci-e2e-test

horizontal-pod-autoscaler

message-dumper-predictor

FailedComputeMetricsReplicas

 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

kserve-ci-e2e-test

horizontal-pod-autoscaler

message-dumper-predictor

FailedGetResourceMetric

 failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-logger-predictor

FailedComputeMetricsReplicas

 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-logger-predictor

FailedGetResourceMetric

 failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x10)

kserve-ci-e2e-test

v1beta1Controllers

isvc-logger

VirtualServiceCRDNotFound

 Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true.

kserve-ci-e2e-test

v1beta1Controllers

isvc-logger

InferenceServiceReady

 InferenceService [isvc-logger] is Ready
(x4)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-logger-predictor

FailedComputeMetricsReplicas

 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x4)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-logger-predictor

FailedGetResourceMetric

 failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

kserve-ci-e2e-test

deployment-controller

isvc-autogluon-v1-predictor

ScalingReplicaSet

 Scaled up replica set isvc-autogluon-v1-predictor-7bbbbd79cd from 0 to 1

kserve-ci-e2e-test

replicaset-controller

isvc-autogluon-v1-predictor-7bbbbd79cd

SuccessfulCreate

 Created pod: isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb

kserve-ci-e2e-test

kubelet

message-dumper-predictor-6575bcfdfd-v85ks

Killing

 Stopping container kube-rbac-proxy

kserve-ci-e2e-test

kubelet

message-dumper-predictor-6575bcfdfd-v85ks

Killing

 Stopping container kserve-container

kserve-ci-e2e-test

kubelet

isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb

FailedMount

 MountVolume.SetUp failed for volume "proxy-tls" : secret "isvc-autogluon-v1-predictor-serving-cert" not found

kserve-ci-e2e-test

v1beta1Controllers

isvc-autogluon-v1

InternalError

 fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "isvc-autogluon-v1": the object has been modified; please apply your changes to the latest version and try again

kserve-ci-e2e-test

kubelet

isvc-logger-predictor-5d9d8fb8b4-c549m

Killing

 Stopping container kserve-container

kserve-ci-e2e-test

kubelet

isvc-logger-predictor-5d9d8fb8b4-c549m

Killing

 Stopping container agent

kserve-ci-e2e-test

kubelet

isvc-logger-predictor-5d9d8fb8b4-c549m

Killing

 Stopping container kube-rbac-proxy

kserve-ci-e2e-test

v1beta1Controllers

isvc-autogluon-v1

UpdateFailed

 Failed to update status for InferenceService "isvc-autogluon-v1": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "isvc-autogluon-v1": the object has been modified; please apply your changes to the latest version and try again

kserve-ci-e2e-test

kubelet

isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb

Created

 Created container: storage-initializer

kserve-ci-e2e-test

multus

isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb

AddedInterface

 Add eth0 [10.134.0.30/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb

Pulled

 Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb

Started

 Started container storage-initializer

kserve-ci-e2e-test

kubelet

isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb

Pulling

 Pulling image "kserve/autogluonserver:latest"
(x10)

kserve-ci-e2e-test

kubelet

isvc-logger-predictor-5d9d8fb8b4-c549m

Unhealthy

 Readiness probe failed: dial tcp 10.134.0.29:8080: connect: connection refused
(x11)

kserve-ci-e2e-test

kubelet

isvc-logger-predictor-5d9d8fb8b4-c549m

Unhealthy

 Readiness probe failed: HTTP probe failed with statuscode: 503
(x4)

kserve-ci-e2e-test

kubelet

isvc-logger-predictor-5d9d8fb8b4-c549m

Unhealthy

 Readiness probe failed: Get "https://10.134.0.29:8643/healthz": dial tcp 10.134.0.29:8643: connect: connection refused

kserve-ci-e2e-test

kubelet

isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb

Created

 Created container: kube-rbac-proxy

kserve-ci-e2e-test

kubelet

isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb

Pulled

 Successfully pulled image "kserve/autogluonserver:latest" in 2m20.229s (2m20.229s including waiting). Image size: 8939191061 bytes.

kserve-ci-e2e-test

kubelet

isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb

Started

 Started container kube-rbac-proxy

kserve-ci-e2e-test

kubelet

isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb

Created

 Created container: kserve-container

kserve-ci-e2e-test

kubelet

isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb

Pulled

 Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine

kserve-ci-e2e-test

kubelet

isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb

Started

 Started container kserve-container
(x10)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-autogluon-v1-predictor

FailedComputeMetricsReplicas

 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x10)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-autogluon-v1-predictor

FailedGetResourceMetric

 failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

kserve-ci-e2e-test

kubelet

isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb

Unhealthy

 Readiness probe failed: dial tcp 10.134.0.30:8080: connect: connection refused

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-autogluon-v1-predictor

FailedComputeMetricsReplicas

 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

kserve-ci-e2e-test

v1beta1Controllers

isvc-autogluon-v1

InferenceServiceReady

 InferenceService [isvc-autogluon-v1] is Ready

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-autogluon-v1-predictor

FailedGetResourceMetric

 failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x12)

kserve-ci-e2e-test

v1beta1Controllers

isvc-autogluon-v1

VirtualServiceCRDNotFound

 Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true.

kserve-ci-e2e-test

kubelet

isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb

Killing

 Stopping container kserve-container

kserve-ci-e2e-test

kubelet

isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb

Killing

 Stopping container kube-rbac-proxy

kserve-ci-e2e-test

v1beta1Controllers

isvc-autogluon-v2

UpdateFailed

 Failed to update status for InferenceService "isvc-autogluon-v2": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "isvc-autogluon-v2": the object has been modified; please apply your changes to the latest version and try again

kserve-ci-e2e-test

deployment-controller

isvc-autogluon-v2-predictor

ScalingReplicaSet

 Scaled up replica set isvc-autogluon-v2-predictor-67b849d456 from 0 to 1

kserve-ci-e2e-test

replicaset-controller

isvc-autogluon-v2-predictor-67b849d456

SuccessfulCreate

 Created pod: isvc-autogluon-v2-predictor-67b849d456-xg4r6

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-predictor-67b849d456-xg4r6

Started

 Started container storage-initializer

kserve-ci-e2e-test

multus

isvc-autogluon-v2-predictor-67b849d456-xg4r6

AddedInterface

 Add eth0 [10.134.0.31/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-predictor-67b849d456-xg4r6

Created

 Created container: storage-initializer

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-predictor-67b849d456-xg4r6

Pulled

 Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

v1beta1Controllers

isvc-autogluon-v2

InternalError

 fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "isvc-autogluon-v2": the object has been modified; please apply your changes to the latest version and try again

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-predictor-67b849d456-xg4r6

Created

 Created container: kube-rbac-proxy

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-predictor-67b849d456-xg4r6

Pulled

 Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-predictor-67b849d456-xg4r6

Started

 Started container kube-rbac-proxy

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-predictor-67b849d456-xg4r6

Started

 Started container kserve-container

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-predictor-67b849d456-xg4r6

Created

 Created container: kserve-container

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-predictor-67b849d456-xg4r6

Pulled

 Container image "kserve/autogluonserver:latest" already present on machine
(x3)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-autogluon-v2-predictor

FailedComputeMetricsReplicas

 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-autogluon-v2-predictor

FailedGetResourceMetric

 failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-autogluon-v2-predictor

FailedGetResourceMetric

 failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x5)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-autogluon-v2-predictor

FailedComputeMetricsReplicas

 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x12)

kserve-ci-e2e-test

v1beta1Controllers

isvc-autogluon-v2

VirtualServiceCRDNotFound

 Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true.

kserve-ci-e2e-test

v1beta1Controllers

isvc-autogluon-v2

InferenceServiceReady

 InferenceService [isvc-autogluon-v2] is Ready

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-predictor-67b849d456-xg4r6

Killing

 Stopping container kube-rbac-proxy

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-predictor-67b849d456-xg4r6

Killing

 Stopping container kserve-container
(x2)

kserve-ci-e2e-test

v1beta1Controllers

isvc-autogluon-v2-variants

InternalError

 fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "isvc-autogluon-v2-variants": the object has been modified; please apply your changes to the latest version and try again

kserve-ci-e2e-test

deployment-controller

isvc-autogluon-v2-variants-predictor

ScalingReplicaSet

 Scaled up replica set isvc-autogluon-v2-variants-predictor-c6767c94c from 0 to 1
(x2)

kserve-ci-e2e-test

v1beta1Controllers

isvc-autogluon-v2-variants

UpdateFailed

 Failed to update status for InferenceService "isvc-autogluon-v2-variants": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "isvc-autogluon-v2-variants": the object has been modified; please apply your changes to the latest version and try again

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw

Pulled

 Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

multus

isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw

AddedInterface

 Add eth0 [10.134.0.32/23] from ovn-kubernetes

kserve-ci-e2e-test

replicaset-controller

isvc-autogluon-v2-variants-predictor-c6767c94c

SuccessfulCreate

 Created pod: isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw

Created

 Created container: storage-initializer

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw

Started

 Started container storage-initializer

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw

Created

 Created container: kube-rbac-proxy

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw

Started

 Started container kube-rbac-proxy

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw

Pulled

 Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw

Started

 Started container kserve-container

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw

Created

 Created container: kserve-container

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw

Pulled

 Container image "kserve/autogluonserver:latest" already present on machine
(x3)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-autogluon-v2-variants-predictor

FailedGetResourceMetric

 failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-autogluon-v2-variants-predictor

FailedComputeMetricsReplicas

 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-autogluon-v2-variants-predictor

FailedGetResourceMetric

 failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x5)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-autogluon-v2-variants-predictor

FailedComputeMetricsReplicas

 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

kserve-ci-e2e-test

v1beta1Controllers

isvc-autogluon-v2-variants

InferenceServiceReady

 InferenceService [isvc-autogluon-v2-variants] is Ready
(x11)

kserve-ci-e2e-test

v1beta1Controllers

isvc-autogluon-v2-variants

VirtualServiceCRDNotFound

 Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true.

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw

Killing

 Stopping container kserve-container

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw

Killing

 Stopping container kube-rbac-proxy

kserve-ci-e2e-test

deployment-controller

isvc-autogluon-v2-noslash-predictor

ScalingReplicaSet

 Scaled up replica set isvc-autogluon-v2-noslash-predictor-9dcfbbb84 from 0 to 1
(x2)

kserve-ci-e2e-test

v1beta1Controllers

isvc-autogluon-v2-noslash

InternalError

 fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "isvc-autogluon-v2-noslash": the object has been modified; please apply your changes to the latest version and try again
(x2)

kserve-ci-e2e-test

v1beta1Controllers

isvc-autogluon-v2-noslash

UpdateFailed

 Failed to update status for InferenceService "isvc-autogluon-v2-noslash": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "isvc-autogluon-v2-noslash": the object has been modified; please apply your changes to the latest version and try again

kserve-ci-e2e-test

replicaset-controller

isvc-autogluon-v2-noslash-predictor-9dcfbbb84

SuccessfulCreate

 Created pod: isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn

Created

 Created container: storage-initializer

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn

Started

 Started container storage-initializer

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn

Pulled

 Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

multus

isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn

AddedInterface

 Add eth0 [10.134.0.33/23] from ovn-kubernetes

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn

Pulled

 Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn

Started

 Started container kube-rbac-proxy

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn

Created

 Created container: kube-rbac-proxy

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn

Started

 Started container kserve-container

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn

Created

 Created container: kserve-container

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn

Pulled

 Container image "kserve/autogluonserver:latest" already present on machine
(x3)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-autogluon-v2-noslash-predictor

FailedGetResourceMetric

 failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-autogluon-v2-noslash-predictor

FailedComputeMetricsReplicas

 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-autogluon-v2-noslash-predictor

FailedGetResourceMetric

 failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-autogluon-v2-noslash-predictor

FailedComputeMetricsReplicas

 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x12)

kserve-ci-e2e-test

v1beta1Controllers

isvc-autogluon-v2-noslash

VirtualServiceCRDNotFound

 Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true.

kserve-ci-e2e-test

v1beta1Controllers

isvc-autogluon-v2-noslash

InferenceServiceReady

 InferenceService [isvc-autogluon-v2-noslash] is Ready

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn

Killing

 Stopping container kube-rbac-proxy

kserve-ci-e2e-test

kubelet

isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn

Killing

 Stopping container kserve-container

kserve-ci-e2e-test

replicaset-controller

isvc-autogluon-ts-v1-predictor-89d67dbf7

SuccessfulCreate

 Created pod: isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt
(x2)

kserve-ci-e2e-test

v1beta1Controllers

isvc-autogluon-ts-v1

InternalError

 fails to update InferenceService status: Operation cannot be fulfilled on inferenceservices.serving.kserve.io "isvc-autogluon-ts-v1": the object has been modified; please apply your changes to the latest version and try again
(x2)

kserve-ci-e2e-test

v1beta1Controllers

isvc-autogluon-ts-v1

UpdateFailed

 Failed to update status for InferenceService "isvc-autogluon-ts-v1": Operation cannot be fulfilled on inferenceservices.serving.kserve.io "isvc-autogluon-ts-v1": the object has been modified; please apply your changes to the latest version and try again

kserve-ci-e2e-test

kubelet

isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt

Started

 Started container storage-initializer

kserve-ci-e2e-test

multus

isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt

AddedInterface

 Add eth0 [10.134.0.34/23] from ovn-kubernetes

kserve-ci-e2e-test

deployment-controller

isvc-autogluon-ts-v1-predictor

ScalingReplicaSet

 Scaled up replica set isvc-autogluon-ts-v1-predictor-89d67dbf7 from 0 to 1

kserve-ci-e2e-test

kubelet

isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt

Created

 Created container: storage-initializer

kserve-ci-e2e-test

kubelet

isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt

Pulled

 Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:38f70cb77701ee00560ecdde553dd84342a4d8f7cb0779aa5b508807d9895bf1" already present on machine

kserve-ci-e2e-test

kubelet

isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt

Started

 Started container kube-rbac-proxy

kserve-ci-e2e-test

kubelet

isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt

Created

 Created container: kube-rbac-proxy

kserve-ci-e2e-test

kubelet

isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt

Pulled

 Container image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3" already present on machine

kserve-ci-e2e-test

kubelet

isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt

Started

 Started container kserve-container

kserve-ci-e2e-test

kubelet

isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt

Created

 Created container: kserve-container

kserve-ci-e2e-test

kubelet

isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt

Pulled

 Container image "kserve/autogluonserver:latest" already present on machine

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-autogluon-ts-v1-predictor

FailedGetResourceMetric

 failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-autogluon-ts-v1-predictor

FailedComputeMetricsReplicas

 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

kserve-ci-e2e-test

kubelet

isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt

Unhealthy

 Readiness probe failed: dial tcp 10.134.0.34:8080: connect: connection refused
(x11)

kserve-ci-e2e-test

v1beta1Controllers

isvc-autogluon-ts-v1

VirtualServiceCRDNotFound

 Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true.

kserve-ci-e2e-test

v1beta1Controllers

isvc-autogluon-ts-v1

InferenceServiceReady

 InferenceService [isvc-autogluon-ts-v1] is Ready

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-autogluon-ts-v1-predictor

FailedGetResourceMetric

 failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

kserve-ci-e2e-test

kubelet

isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt

Killing

 Stopping container kserve-container

kserve-ci-e2e-test

kubelet

isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt

Killing

 Stopping container kube-rbac-proxy

kserve-ci-e2e-test

horizontal-pod-autoscaler

isvc-autogluon-ts-v1-predictor

FailedComputeMetricsReplicas

 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)