LAST_SEEN | FIRST_SEEN | NAME | SUBOBJECT | TYPE | REASON | MESSAGE | 2026-11-02 23:08:47 | 2026-11-02 23:08:47 | pvc-f8t8h.18e45d1c3b554af2 | - | Normal | ExternalProvisioning | Waiting for a volume to be created either by the external provisioner 'example.com/nfs' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered. | 2026-11-02 23:08:47 | 2026-11-02 23:08:47 | pvc-f8t8h.18e45d1c3bb6f4db | - | Normal | Provisioning | External provisioner is provisioning volume for claim "test-ns-hw8fl/pvc-f8t8h" | 2026-11-02 23:08:47 | 2026-11-02 23:08:47 | pvc-f8t8h.18e45d1c3cbaf3ca | - | Normal | ProvisioningSucceeded | Successfully provisioned volume pvc-65d6e616-3c41-4408-952f-4afd75f421fa | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | test-fashion-mnist-trainjob-z2d5d-dataset-initializer-0-0-4tztk.18e45d1c78411f5a | - | Normal | Scheduled | Successfully assigned test-ns-hw8fl/test-fashion-mnist-trainjob-z2d5d-dataset-initializer-0-0-4tztk to ip-10-0-131-91.us-east-2.compute.internal | 2026-11-02 23:08:49 | 2026-11-02 23:08:49 | test-fashion-mnist-trainjob-z2d5d-dataset-initializer-0-0-4tztk.18e45d1c9a4ef83c | - | Normal | AddedInterface | Add eth0 [10.133.0.44/23] from ovn-kubernetes | 2026-11-02 23:08:49 | 2026-11-02 23:08:49 | test-fashion-mnist-trainjob-z2d5d-dataset-initializer-0-0-4tztk.18e45d1c9bdd1b44 | spec.containers{dataset-initializer} | Normal | Pulled | Container image "quay.io/opendatahub/odh-th-torch-cpu-py312:odh-stable" already present on machine and can be accessed by the pod | 2026-11-02 23:08:49 | 2026-11-02 23:08:49 | test-fashion-mnist-trainjob-z2d5d-dataset-initializer-0-0-4tztk.18e45d1c9e9a0df2 | spec.containers{dataset-initializer} | Normal | Created | Container created | 2026-11-02 23:08:49 | 2026-11-02 23:08:49 | test-fashion-mnist-trainjob-z2d5d-dataset-initializer-0-0-4tztk.18e45d1c9f0d932a | spec.containers{dataset-initializer} | Normal | Started | Container started | 2026-11-02 23:08:47 | 2026-11-02 23:08:47 | test-fashion-mnist-trainjob-z2d5d-dataset-initializer-0.18e45d1c5ade88d4 | - | Normal | Suspended | Job suspended | 2026-11-02 23:08:48 | 2026-11-02 23:08:48 | test-fashion-mnist-trainjob-z2d5d-dataset-initializer-0.18e45d1c77ba56f5 | - | Normal | SuccessfulCreate | Created pod: test-fashion-mnist-trainjob-z2d5d-dataset-initializer-0-0-4tztk | 2026-11-02 23:08:48 | 2026-11-02 23:08:48 | test-fashion-mnist-trainjob-z2d5d-dataset-initializer-0.18e45d1c77bb0494 | - | Normal | Resumed | Job resumed | 2026-11-02 23:09:12 | 2026-11-02 23:09:12 | test-fashion-mnist-trainjob-z2d5d-dataset-initializer-0.18e45d21f40d0c73 | - | Normal | Completed | Job completed | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | test-fashion-mnist-trainjob-z2d5d-node-0-0-b4zw5.18e45d21f76d7023 | - | Normal | Scheduled | Successfully assigned test-ns-hw8fl/test-fashion-mnist-trainjob-z2d5d-node-0-0-b4zw5 to ip-10-0-131-91.us-east-2.compute.internal | 2026-11-02 23:09:12 | 2026-11-02 23:09:12 | test-fashion-mnist-trainjob-z2d5d-node-0-0-b4zw5.18e45d2219b4a5f2 | - | Normal | AddedInterface | Add eth0 [10.133.0.45/23] from ovn-kubernetes | 2026-11-02 23:09:12 | 2026-11-02 23:09:12 | test-fashion-mnist-trainjob-z2d5d-node-0-0-b4zw5.18e45d221b4836e3 | spec.containers{node} | Normal | Pulled | Container image "quay.io/opendatahub/odh-th-torch-cpu-py312:odh-stable" already present on machine and can be accessed by the pod | 2026-11-02 23:09:12 | 2026-11-02 23:09:12 | test-fashion-mnist-trainjob-z2d5d-node-0-0-b4zw5.18e45d221e6b1884 | spec.containers{node} | Normal | Created | Container created | 2026-11-02 23:09:12 | 2026-11-02 23:09:12 | test-fashion-mnist-trainjob-z2d5d-node-0-0-b4zw5.18e45d221ee3a619 | spec.containers{node} | Normal | Started | Container started | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | test-fashion-mnist-trainjob-z2d5d-node-0-1-q8mk9.18e45d21f9369919 | - | Normal | Scheduled | Successfully assigned test-ns-hw8fl/test-fashion-mnist-trainjob-z2d5d-node-0-1-q8mk9 to ip-10-0-139-99.us-east-2.compute.internal | 2026-11-02 23:09:12 | 2026-11-02 23:09:12 | test-fashion-mnist-trainjob-z2d5d-node-0-1-q8mk9.18e45d221b63541b | - | Normal | AddedInterface | Add eth0 [10.134.0.57/23] from ovn-kubernetes | 2026-11-02 23:09:12 | 2026-11-02 23:09:12 | test-fashion-mnist-trainjob-z2d5d-node-0-1-q8mk9.18e45d221cf3143e | spec.containers{node} | Normal | Pulled | Container image "quay.io/opendatahub/odh-th-torch-cpu-py312:odh-stable" already present on machine and can be accessed by the pod | 2026-11-02 23:09:12 | 2026-11-02 23:09:12 | test-fashion-mnist-trainjob-z2d5d-node-0-1-q8mk9.18e45d221fd77d99 | spec.containers{node} | Normal | Created | Container created | 2026-11-02 23:09:12 | 2026-11-02 23:09:12 | test-fashion-mnist-trainjob-z2d5d-node-0-1-q8mk9.18e45d222053296c | spec.containers{node} | Normal | Started | Container started | 2026-11-02 23:09:12 | 2026-11-02 23:09:12 | test-fashion-mnist-trainjob-z2d5d-node-0.18e45d21f6f86cb4 | - | Normal | SuccessfulCreate | Created pod: test-fashion-mnist-trainjob-z2d5d-node-0-0-b4zw5 | 2026-11-02 23:09:12 | 2026-11-02 23:09:12 | test-fashion-mnist-trainjob-z2d5d-node-0.18e45d21f8c7269d | - | Normal | SuccessfulCreate | Created pod: test-fashion-mnist-trainjob-z2d5d-node-0-1-q8mk9 | 2026-11-02 23:09:57 | 2026-11-02 23:09:57 | test-fashion-mnist-trainjob-z2d5d-node-0.18e45d2c7e832c72 | - | Normal | Completed | Job completed | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | test-fashion-mnist-trainjob-z2d5d.18e45d1c59646ff4 | - | Normal | CreatedWorkload | Created Workload: test-ns-hw8fl/trainjob-test-fashion-mnist-trainjob-z2d5d-40475 | 2026-11-02 23:08:47 | 2026-11-02 23:08:47 | test-fashion-mnist-trainjob-z2d5d.18e45d1c5beb2c78 | - | Normal | SuspendedJobs | jobset is suspended | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | test-fashion-mnist-trainjob-z2d5d.18e45d1c628b184d | - | Normal | Started | Admitted by clusterQueue cq-nc8dx | 2026-11-02 23:08:48 | 2026-11-02 23:08:48 | test-fashion-mnist-trainjob-z2d5d.18e45d1c66cd2b37 | - | Normal | ResumeJobs | jobset is resumed | 2026-11-02 23:09:57 | 2026-11-02 23:09:57 | test-fashion-mnist-trainjob-z2d5d.18e45d2c7f8348d6 | - | Normal | AllJobsCompleted | jobset completed successfully | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | test-fashion-mnist-trainjob-z2d5d.18e45d2c835e5ac4 | - | Normal | FinishedWorkload | Workload 'test-ns-hw8fl/trainjob-test-fashion-mnist-trainjob-z2d5d-40475' is declared finished | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | trainjob-test-fashion-mnist-trainjob-z2d5d-40475.18e45d1c5b07ab53 | - | Normal | QuotaReserved | Quota reserved in ClusterQueue cq-nc8dx, wait time since queued was 1s | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | trainjob-test-fashion-mnist-trainjob-z2d5d-40475.18e45d1c5b0845a5 | - | Normal | Admitted | Admitted by ClusterQueue cq-nc8dx, wait time since reservation was 1s |