LAST_SEEN | FIRST_SEEN | NAME | SUBOBJECT | TYPE | REASON | MESSAGE | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | test-preemption-trainjob-7tplp-node-0-0-qqz9b.18e33d78b61477f8 | - | Normal | Scheduled | Successfully assigned test-ns-nnx5b/test-preemption-trainjob-7tplp-node-0-0-qqz9b to ip-10-0-135-30.ec2.internal | 2026-10-30 07:17:45 | 2026-10-30 07:17:45 | test-preemption-trainjob-7tplp-node-0-0-qqz9b.18e33d78d904b6b3 | - | Normal | AddedInterface | Add eth0 [10.133.0.40/23] from ovn-kubernetes | 2026-10-30 07:17:45 | 2026-10-30 07:17:45 | test-preemption-trainjob-7tplp-node-0-0-qqz9b.18e33d78dbaf17d5 | spec.containers{node} | Normal | Pulled | Container image "quay.io/opendatahub/odh-th-torch-cuda-py312:odh-stable" already present on machine and can be accessed by the pod | 2026-10-30 07:17:45 | 2026-10-30 07:17:45 | test-preemption-trainjob-7tplp-node-0-0-qqz9b.18e33d78deb5e551 | spec.containers{node} | Normal | Created | Container created | 2026-10-30 07:17:45 | 2026-10-30 07:17:45 | test-preemption-trainjob-7tplp-node-0-0-qqz9b.18e33d78df235381 | spec.containers{node} | Normal | Started | Container started | 2026-10-30 07:17:47 | 2026-10-30 07:17:44 | test-preemption-trainjob-7tplp-node-0.18e33d789a8a4897 | - | Normal | Suspended | Job suspended | 2026-10-30 07:17:45 | 2026-10-30 07:17:45 | test-preemption-trainjob-7tplp-node-0.18e33d78b589feff | - | Normal | SuccessfulCreate | Created pod: test-preemption-trainjob-7tplp-node-0-0-qqz9b | 2026-10-30 07:17:45 | 2026-10-30 07:17:45 | test-preemption-trainjob-7tplp-node-0.18e33d78b58a55db | - | Normal | Resumed | Job resumed | 2026-10-30 07:17:47 | 2026-10-30 07:17:47 | test-preemption-trainjob-7tplp-node-0.18e33d792133fcab | - | Normal | SuccessfulDelete | Deleted pod: test-preemption-trainjob-7tplp-node-0-0-qqz9b | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | test-preemption-trainjob-7tplp.18e33d78993c2a2d | - | Normal | CreatedWorkload | Created Workload: test-ns-nnx5b/trainjob-test-preemption-trainjob-7tplp-f49c8 | 2026-10-30 07:17:47 | 2026-10-30 07:17:44 | test-preemption-trainjob-7tplp.18e33d789d72ac20 | - | Normal | SuspendedJobs | jobset is suspended | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | test-preemption-trainjob-7tplp.18e33d78ae1bc9fd | - | Normal | Started | Admitted by clusterQueue cq-mvgn7 | 2026-10-30 07:17:45 | 2026-10-30 07:17:45 | test-preemption-trainjob-7tplp.18e33d78b3f57abf | - | Normal | ResumeJobs | jobset is resumed | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | test-preemption-trainjob-7tplp.18e33d791b2f0c7e | - | Normal | Stopped | The workload is deactivated | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | test-preemption-trainjob-7tplp.18e33d79320908f8 | - | Warning | TrainJobResourcesCreationFailed | TrainJob resources reconciliation failed: JobSet.jobset.x-k8s.io "test-preemption-trainjob-7tplp" is invalid: [spec.replicatedJobs[0].template.spec.template.metadata: Invalid value: "null": spec.replicatedJobs[0].template.spec.template.metadata in body must be of type object: "null", : Invalid value: null: some validation rules were not checked because the object was invalid; correct the existing errors to complete validation] | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | trainjob-test-preemption-trainjob-7tplp-f49c8.18e33d789b8ede21 | - | Normal | QuotaReserved | Quota reserved in ClusterQueue cq-mvgn7, wait time since queued was 1s | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | trainjob-test-preemption-trainjob-7tplp-f49c8.18e33d789b8f6cfa | - | Normal | Admitted | Admitted by ClusterQueue cq-mvgn7, wait time since reservation was 1s | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | trainjob-test-preemption-trainjob-7tplp-f49c8.18e33d7914a1bded | - | Normal | EvictedDueToDeactivated | The workload is deactivated |