Events

Time	Namespace	Component	RelatedObject	Reason	Message
03:53:21	opendatahub		kubeflow-trainer-controller-manager-679c94ffcd-bcxj6	Scheduled	Successfully assigned opendatahub/kubeflow-trainer-controller-manager-679c94ffcd-bcxj6 to ip-10-0-143-175.ec2.internal
03:53:23	opendatahub		kubeflow-trainer-controller-manager-679c94ffcd-zstk5	Scheduled	Successfully assigned opendatahub/kubeflow-trainer-controller-manager-679c94ffcd-zstk5 to ip-10-0-132-57.ec2.internal
03:53:18	opendatahub		kubeflow-trainer-controller-manager-679c94ffcd-88kg4	Scheduled	Successfully assigned opendatahub/kubeflow-trainer-controller-manager-679c94ffcd-88kg4 to ip-10-0-143-175.ec2.internal
03:52:51	opendatahub		kubeflow-trainer-controller-manager-7b4fcbc79d-t75c7	Scheduled	Successfully assigned opendatahub/kubeflow-trainer-controller-manager-7b4fcbc79d-t75c7 to ip-10-0-132-57.ec2.internal
03:53:29	opendatahub		kubeflow-trainer-controller-manager-679c94ffcd-m65gs	Scheduled	Successfully assigned opendatahub/kubeflow-trainer-controller-manager-679c94ffcd-m65gs to ip-10-0-143-175.ec2.internal
03:53:25	opendatahub		kubeflow-trainer-controller-manager-679c94ffcd-8tgvs	Scheduled	Successfully assigned opendatahub/kubeflow-trainer-controller-manager-679c94ffcd-8tgvs to ip-10-0-143-175.ec2.internal
03:53:27	opendatahub		kubeflow-trainer-controller-manager-679c94ffcd-h724d	Scheduled	Successfully assigned opendatahub/kubeflow-trainer-controller-manager-679c94ffcd-h724d to ip-10-0-143-175.ec2.internal
03:52:51	opendatahub	replicaset-controller	kubeflow-trainer-controller-manager-7b4fcbc79d	SuccessfulCreate	Created pod: kubeflow-trainer-controller-manager-7b4fcbc79d-t75c7
03:52:51	opendatahub	deployment-controller	kubeflow-trainer-controller-manager	ScalingReplicaSet	Scaled up replica set kubeflow-trainer-controller-manager-7b4fcbc79d from 0 to 1
03:52:52	opendatahub	multus	kubeflow-trainer-controller-manager-7b4fcbc79d-t75c7	AddedInterface	Add eth0 [10.133.0.21/23] from ovn-kubernetes
03:52:52	opendatahub	kubelet	kubeflow-trainer-controller-manager-7b4fcbc79d-t75c7	Pulling	Pulling image "quay.io/opendatahub/trainer:odh-3.5-ea1"
03:52:54	opendatahub	kubelet	kubeflow-trainer-controller-manager-7b4fcbc79d-t75c7	Started	Started container manager
03:52:54	opendatahub	kubelet	kubeflow-trainer-controller-manager-7b4fcbc79d-t75c7	Pulled	Successfully pulled image "quay.io/opendatahub/trainer:odh-3.5-ea1" in 2.38s (2.38s including waiting). Image size: 182722760 bytes.
03:52:54	opendatahub	kubeflow-trainer-controller-manager-7b4fcbc79d-t75c7_e3a976c6-fa67-4ab6-ad42-1223988ad69c	trainer.kubeflow.org	LeaderElection	kubeflow-trainer-controller-manager-7b4fcbc79d-t75c7_e3a976c6-fa67-4ab6-ad42-1223988ad69c became leader
03:52:54	opendatahub	kubelet	kubeflow-trainer-controller-manager-7b4fcbc79d-t75c7	Created	Created container: manager
03:53:18	opendatahub	replicaset-controller	kubeflow-trainer-controller-manager-679c94ffcd	SuccessfulCreate	Created pod: kubeflow-trainer-controller-manager-679c94ffcd-88kg4
03:53:19	opendatahub	multus	kubeflow-trainer-controller-manager-679c94ffcd-88kg4	AddedInterface	Add eth0 [10.132.0.29/23] from ovn-kubernetes
03:53:19	opendatahub	kubelet	kubeflow-trainer-controller-manager-679c94ffcd-88kg4	Pulling	Pulling image "quay.io/opendatahub/trainer:odh-3.5"
03:53:19	opendatahub	kubelet	kubeflow-trainer-controller-manager-679c94ffcd-88kg4	Failed	Failed to pull image "quay.io/opendatahub/trainer:odh-3.5": unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/opendatahub/trainer:odh-3.5: reading manifest odh-3.5 in quay.io/opendatahub/trainer: manifest unknown; artifact err: get manifest: build image source: reading manifest odh-3.5 in quay.io/opendatahub/trainer: manifest unknown
03:53:19	opendatahub	kubelet	kubeflow-trainer-controller-manager-679c94ffcd-88kg4	Failed	Error: ErrImagePull
03:53:20	opendatahub	replicaset-controller	kubeflow-trainer-controller-manager-679c94ffcd	SuccessfulDelete	Deleted pod: kubeflow-trainer-controller-manager-679c94ffcd-88kg4
03:53:21	opendatahub	replicaset-controller	kubeflow-trainer-controller-manager-679c94ffcd	SuccessfulCreate	Created pod: kubeflow-trainer-controller-manager-679c94ffcd-bcxj6
03:53:21 (x2)	opendatahub	kubelet	kubeflow-trainer-controller-manager-679c94ffcd-88kg4	BackOff	Back-off pulling image "quay.io/opendatahub/trainer:odh-3.5"
03:53:21 (x2)	opendatahub	kubelet	kubeflow-trainer-controller-manager-679c94ffcd-88kg4	Failed	Error: ImagePullBackOff
03:53:22	opendatahub	kubelet	kubeflow-trainer-controller-manager-679c94ffcd-bcxj6	Failed	Error: ErrImagePull
03:53:22	opendatahub	replicaset-controller	kubeflow-trainer-controller-manager-679c94ffcd	SuccessfulDelete	Deleted pod: kubeflow-trainer-controller-manager-679c94ffcd-bcxj6
03:53:22	opendatahub	kubelet	kubeflow-trainer-controller-manager-679c94ffcd-bcxj6	Failed	Failed to pull image "quay.io/opendatahub/trainer:odh-3.5": unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/opendatahub/trainer:odh-3.5: reading manifest odh-3.5 in quay.io/opendatahub/trainer: manifest unknown; artifact err: get manifest: build image source: reading manifest odh-3.5 in quay.io/opendatahub/trainer: manifest unknown
03:53:22	opendatahub	kubelet	kubeflow-trainer-controller-manager-679c94ffcd-bcxj6	Pulling	Pulling image "quay.io/opendatahub/trainer:odh-3.5"
03:53:22	opendatahub	multus	kubeflow-trainer-controller-manager-679c94ffcd-bcxj6	AddedInterface	Add eth0 [10.132.0.30/23] from ovn-kubernetes
03:53:23	opendatahub	replicaset-controller	kubeflow-trainer-controller-manager-679c94ffcd	SuccessfulCreate	Created pod: kubeflow-trainer-controller-manager-679c94ffcd-zstk5
03:53:24	opendatahub	multus	kubeflow-trainer-controller-manager-679c94ffcd-zstk5	AddedInterface	Add eth0 [10.133.0.23/23] from ovn-kubernetes
03:53:24	opendatahub	kubelet	kubeflow-trainer-controller-manager-679c94ffcd-zstk5	Pulling	Pulling image "quay.io/opendatahub/trainer:odh-3.5"
03:53:24	opendatahub	kubelet	kubeflow-trainer-controller-manager-679c94ffcd-zstk5	Failed	Failed to pull image "quay.io/opendatahub/trainer:odh-3.5": unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/opendatahub/trainer:odh-3.5: reading manifest odh-3.5 in quay.io/opendatahub/trainer: manifest unknown; artifact err: get manifest: build image source: reading manifest odh-3.5 in quay.io/opendatahub/trainer: manifest unknown
03:53:24	opendatahub	kubelet	kubeflow-trainer-controller-manager-679c94ffcd-zstk5	Failed	Error: ErrImagePull
03:53:24	opendatahub	replicaset-controller	kubeflow-trainer-controller-manager-679c94ffcd	SuccessfulDelete	Deleted pod: kubeflow-trainer-controller-manager-679c94ffcd-zstk5
03:53:25	opendatahub	replicaset-controller	kubeflow-trainer-controller-manager-679c94ffcd	SuccessfulCreate	Created pod: kubeflow-trainer-controller-manager-679c94ffcd-8tgvs
03:53:26	opendatahub	replicaset-controller	kubeflow-trainer-controller-manager-679c94ffcd	SuccessfulDelete	Deleted pod: kubeflow-trainer-controller-manager-679c94ffcd-8tgvs
03:53:26	opendatahub	kubelet	kubeflow-trainer-controller-manager-679c94ffcd-8tgvs	Failed	Error: ErrImagePull
03:53:26	opendatahub	kubelet	kubeflow-trainer-controller-manager-679c94ffcd-8tgvs	Failed	Failed to pull image "quay.io/opendatahub/trainer:odh-3.5": unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/opendatahub/trainer:odh-3.5: reading manifest odh-3.5 in quay.io/opendatahub/trainer: manifest unknown; artifact err: get manifest: build image source: reading manifest odh-3.5 in quay.io/opendatahub/trainer: manifest unknown
03:53:26	opendatahub	kubelet	kubeflow-trainer-controller-manager-679c94ffcd-8tgvs	Pulling	Pulling image "quay.io/opendatahub/trainer:odh-3.5"
03:53:26	opendatahub	multus	kubeflow-trainer-controller-manager-679c94ffcd-8tgvs	AddedInterface	Add eth0 [10.132.0.31/23] from ovn-kubernetes
03:53:27	opendatahub	replicaset-controller	kubeflow-trainer-controller-manager-679c94ffcd	SuccessfulCreate	Created pod: kubeflow-trainer-controller-manager-679c94ffcd-h724d
03:53:28	opendatahub	kubelet	kubeflow-trainer-controller-manager-679c94ffcd-h724d	Failed	Failed to pull image "quay.io/opendatahub/trainer:odh-3.5": unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/opendatahub/trainer:odh-3.5: reading manifest odh-3.5 in quay.io/opendatahub/trainer: manifest unknown; artifact err: get manifest: build image source: reading manifest odh-3.5 in quay.io/opendatahub/trainer: manifest unknown
03:53:28	opendatahub	replicaset-controller	kubeflow-trainer-controller-manager-679c94ffcd	SuccessfulDelete	Deleted pod: kubeflow-trainer-controller-manager-679c94ffcd-h724d
03:53:28	opendatahub	multus	kubeflow-trainer-controller-manager-679c94ffcd-h724d	AddedInterface	Add eth0 [10.132.0.32/23] from ovn-kubernetes
03:53:28 (x5)	opendatahub	deployment-controller	kubeflow-trainer-controller-manager	ScalingReplicaSet	Scaled down replica set kubeflow-trainer-controller-manager-679c94ffcd from 1 to 0
03:53:28	opendatahub	kubelet	kubeflow-trainer-controller-manager-679c94ffcd-h724d	Pulling	Pulling image "quay.io/opendatahub/trainer:odh-3.5"
03:53:28	opendatahub	kubelet	kubeflow-trainer-controller-manager-679c94ffcd-h724d	Failed	Error: ErrImagePull
03:53:29	opendatahub	replicaset-controller	kubeflow-trainer-controller-manager-679c94ffcd	SuccessfulCreate	Created pod: kubeflow-trainer-controller-manager-679c94ffcd-m65gs
03:53:29 (x6)	opendatahub	deployment-controller	kubeflow-trainer-controller-manager	ScalingReplicaSet	Scaled up replica set kubeflow-trainer-controller-manager-679c94ffcd from 0 to 1
03:53:30	opendatahub	multus	kubeflow-trainer-controller-manager-679c94ffcd-m65gs	AddedInterface	Add eth0 [10.132.0.33/23] from ovn-kubernetes
03:56:13 (x5)	opendatahub	kubelet	kubeflow-trainer-controller-manager-679c94ffcd-m65gs	Pulling	Pulling image "quay.io/opendatahub/trainer:odh-3.5"
03:56:13 (x5)	opendatahub	kubelet	kubeflow-trainer-controller-manager-679c94ffcd-m65gs	Failed	Failed to pull image "quay.io/opendatahub/trainer:odh-3.5": unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/opendatahub/trainer:odh-3.5: reading manifest odh-3.5 in quay.io/opendatahub/trainer: manifest unknown; artifact err: get manifest: build image source: reading manifest odh-3.5 in quay.io/opendatahub/trainer: manifest unknown
03:56:13 (x5)	opendatahub	kubelet	kubeflow-trainer-controller-manager-679c94ffcd-m65gs	Failed	Error: ErrImagePull
03:58:36 (x20)	opendatahub	kubelet	kubeflow-trainer-controller-manager-679c94ffcd-m65gs	Failed	Error: ImagePullBackOff
03:58:36 (x20)	opendatahub	kubelet	kubeflow-trainer-controller-manager-679c94ffcd-m65gs	BackOff	Back-off pulling image "quay.io/opendatahub/trainer:odh-3.5"