LAST_SEEN | FIRST_SEEN | NAME | SUBOBJECT | TYPE | REASON | MESSAGE | 2026-11-02 23:41:10 | 2026-11-02 23:41:10 | pvc-j6gbw.18e45ee08826ff5a | - | Normal | ExternalProvisioning | Waiting for a volume to be created either by the external provisioner 'example.com/nfs' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered. | 2026-11-02 23:41:10 | 2026-11-02 23:41:10 | pvc-j6gbw.18e45ee08d04dff2 | - | Normal | Provisioning | External provisioner is provisioning volume for claim "test-ns-85h9m/pvc-j6gbw" | 2026-11-02 23:41:10 | 2026-11-02 23:41:10 | pvc-j6gbw.18e45ee096dfad2d | - | Normal | ProvisioningSucceeded | Successfully provisioned volume pvc-05d64ca8-4e84-4869-8bcf-3fcdaaf24f31 | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | test-fashion-mnist-trainjob-hq45v-dataset-initializer-0-0-v95ff.18e45ee18ec30ed7 | - | Normal | Scheduled | Successfully assigned test-ns-85h9m/test-fashion-mnist-trainjob-hq45v-dataset-initializer-0-0-v95ff to ip-10-0-136-199.us-west-2.compute.internal | 2026-11-02 23:41:15 | 2026-11-02 23:41:15 | test-fashion-mnist-trainjob-hq45v-dataset-initializer-0-0-v95ff.18e45ee1b2755c5a | - | Normal | AddedInterface | Add eth0 [10.132.0.69/23] from ovn-kubernetes | 2026-11-02 23:41:15 | 2026-11-02 23:41:15 | test-fashion-mnist-trainjob-hq45v-dataset-initializer-0-0-v95ff.18e45ee1bd891f5b | spec.containers{dataset-initializer} | Normal | Pulled | Container image "quay.io/opendatahub/odh-th-torch-cpu-py312:odh-pr-4b5ed8bc6aaf7fd6e67bce4f7bb34a959c2d85ce" already present on machine and can be accessed by the pod | 2026-11-02 23:41:15 | 2026-11-02 23:41:15 | test-fashion-mnist-trainjob-hq45v-dataset-initializer-0-0-v95ff.18e45ee1c15cc041 | spec.containers{dataset-initializer} | Normal | Created | Container created | 2026-11-02 23:41:15 | 2026-11-02 23:41:15 | test-fashion-mnist-trainjob-hq45v-dataset-initializer-0-0-v95ff.18e45ee1c1da3b78 | spec.containers{dataset-initializer} | Normal | Started | Container started | 2026-11-02 23:41:12 | 2026-11-02 23:41:12 | test-fashion-mnist-trainjob-hq45v-dataset-initializer-0.18e45ee1073f4de0 | - | Normal | Suspended | Job suspended | 2026-11-02 23:41:14 | 2026-11-02 23:41:14 | test-fashion-mnist-trainjob-hq45v-dataset-initializer-0.18e45ee18e5a6086 | - | Normal | SuccessfulCreate | Created pod: test-fashion-mnist-trainjob-hq45v-dataset-initializer-0-0-v95ff | 2026-11-02 23:41:14 | 2026-11-02 23:41:14 | test-fashion-mnist-trainjob-hq45v-dataset-initializer-0.18e45ee18e5adf3f | - | Normal | Resumed | Job resumed | 2026-11-02 23:41:45 | 2026-11-02 23:41:45 | test-fashion-mnist-trainjob-hq45v-dataset-initializer-0.18e45ee8d5e943e7 | - | Normal | Completed | Job completed | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | test-fashion-mnist-trainjob-hq45v-node-0-0-hg8sd.18e45ee8e73b4fd9 | - | Normal | Scheduled | Successfully assigned test-ns-85h9m/test-fashion-mnist-trainjob-hq45v-node-0-0-hg8sd to ip-10-0-136-199.us-west-2.compute.internal | 2026-11-02 23:41:46 | 2026-11-02 23:41:46 | test-fashion-mnist-trainjob-hq45v-node-0-0-hg8sd.18e45ee91cd667d2 | - | Normal | AddedInterface | Add eth0 [10.132.0.70/23] from ovn-kubernetes | 2026-11-02 23:41:47 | 2026-11-02 23:41:47 | test-fashion-mnist-trainjob-hq45v-node-0-0-hg8sd.18e45ee9274d428c | spec.containers{node} | Normal | Pulled | Container image "quay.io/opendatahub/odh-th-torch-cpu-py312:odh-pr-4b5ed8bc6aaf7fd6e67bce4f7bb34a959c2d85ce" already present on machine and can be accessed by the pod | 2026-11-02 23:41:47 | 2026-11-02 23:41:47 | test-fashion-mnist-trainjob-hq45v-node-0-0-hg8sd.18e45ee92a2f18fb | spec.containers{node} | Normal | Created | Container created | 2026-11-02 23:41:47 | 2026-11-02 23:41:47 | test-fashion-mnist-trainjob-hq45v-node-0-0-hg8sd.18e45ee92aac09f1 | spec.containers{node} | Normal | Started | Container started | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | test-fashion-mnist-trainjob-hq45v-node-0-1-msqnp.18e45ee8f19fa5a4 | - | Normal | Scheduled | Successfully assigned test-ns-85h9m/test-fashion-mnist-trainjob-hq45v-node-0-1-msqnp to ip-10-0-135-28.us-west-2.compute.internal | 2026-11-02 23:41:47 | 2026-11-02 23:41:47 | test-fashion-mnist-trainjob-hq45v-node-0-1-msqnp.18e45ee939130b4c | - | Normal | AddedInterface | Add eth0 [10.133.0.23/23] from ovn-kubernetes | 2026-11-02 23:41:47 | 2026-11-02 23:41:47 | test-fashion-mnist-trainjob-hq45v-node-0-1-msqnp.18e45ee942a419c4 | spec.containers{node} | Normal | Pulling | Pulling image "quay.io/opendatahub/odh-th-torch-cpu-py312:odh-pr-4b5ed8bc6aaf7fd6e67bce4f7bb34a959c2d85ce" | 2026-11-02 23:42:58 | 2026-11-02 23:42:58 | test-fashion-mnist-trainjob-hq45v-node-0-1-msqnp.18e45ef9b0ac508f | spec.containers{node} | Normal | Pulled | Successfully pulled image "quay.io/opendatahub/odh-th-torch-cpu-py312:odh-pr-4b5ed8bc6aaf7fd6e67bce4f7bb34a959c2d85ce" in 1m10.565s (1m10.565s including waiting). Image size: 7472816286 bytes. | 2026-11-02 23:42:58 | 2026-11-02 23:42:58 | test-fashion-mnist-trainjob-hq45v-node-0-1-msqnp.18e45ef9b60e3ee0 | spec.containers{node} | Normal | Created | Container created | 2026-11-02 23:42:58 | 2026-11-02 23:42:58 | test-fashion-mnist-trainjob-hq45v-node-0-1-msqnp.18e45ef9b75568c0 | spec.containers{node} | Normal | Started | Container started | 2026-11-02 23:41:45 | 2026-11-02 23:41:45 | test-fashion-mnist-trainjob-hq45v-node-0.18e45ee8e6b757f6 | - | Normal | SuccessfulCreate | Created pod: test-fashion-mnist-trainjob-hq45v-node-0-0-hg8sd | 2026-11-02 23:41:46 | 2026-11-02 23:41:46 | test-fashion-mnist-trainjob-hq45v-node-0.18e45ee8f11a171b | - | Normal | SuccessfulCreate | Created pod: test-fashion-mnist-trainjob-hq45v-node-0-1-msqnp | 2026-11-02 23:43:41 | 2026-11-02 23:43:41 | test-fashion-mnist-trainjob-hq45v-node-0.18e45f03c82aa96b | - | Normal | Completed | Job completed | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | test-fashion-mnist-trainjob-hq45v.18e45ee0febbc0e8 | - | Normal | CreatedWorkload | Created Workload: test-ns-85h9m/trainjob-test-fashion-mnist-trainjob-hq45v-e817e | 2026-11-02 23:41:12 | 2026-11-02 23:41:12 | test-fashion-mnist-trainjob-hq45v.18e45ee10c99e766 | - | Normal | SuspendedJobs | jobset is suspended | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | test-fashion-mnist-trainjob-hq45v.18e45ee14b0ef2d6 | - | Normal | Started | Admitted by clusterQueue cq-cxc8z | 2026-11-02 23:41:13 | 2026-11-02 23:41:13 | test-fashion-mnist-trainjob-hq45v.18e45ee1637f5e32 | - | Normal | ResumeJobs | jobset is resumed | 2026-11-02 23:43:41 | 2026-11-02 23:43:41 | test-fashion-mnist-trainjob-hq45v.18e45f03cd7c10c5 | - | Normal | AllJobsCompleted | jobset completed successfully | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | test-fashion-mnist-trainjob-hq45v.18e45f03ebf3c10d | - | Normal | FinishedWorkload | Workload 'test-ns-85h9m/trainjob-test-fashion-mnist-trainjob-hq45v-e817e' is declared finished | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | trainjob-test-fashion-mnist-trainjob-hq45v-e817e.18e45ee105c6056b | - | Normal | QuotaReserved | Quota reserved in ClusterQueue cq-cxc8z, wait time since queued was 1s | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | trainjob-test-fashion-mnist-trainjob-hq45v-e817e.18e45ee105c6bb03 | - | Normal | Admitted | Admitted by ClusterQueue cq-cxc8z, wait time since reservation was 0s |