LAST_SEEN | FIRST_SEEN | NAME | SUBOBJECT | TYPE | REASON | MESSAGE | 2026-10-23 03:57:11 | 2026-10-23 03:57:11 | pvc-g2t6l.18e10c76fe08fda5 | - | Normal | ExternalProvisioning | Waiting for a volume to be created either by the external provisioner 'example.com/nfs' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered. | 2026-10-23 03:57:11 | 2026-10-23 03:57:11 | pvc-g2t6l.18e10c76ff4a07c2 | - | Normal | Provisioning | External provisioner is provisioning volume for claim "test-ns-bglsz/pvc-g2t6l" | 2026-10-23 03:57:11 | 2026-10-23 03:57:11 | pvc-g2t6l.18e10c770243deb8 | - | Normal | ProvisioningSucceeded | Successfully provisioned volume pvc-1b299cbe-9058-4c3d-8a86-25ae0e8628b9 | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | test-fashion-mnist-trainjob-h8djc-dataset-initializer-0-0-r97fj.18e10c7758119a16 | - | Normal | Scheduled | Successfully assigned test-ns-bglsz/test-fashion-mnist-trainjob-h8djc-dataset-initializer-0-0-r97fj to ip-10-0-140-146.ec2.internal | 2026-10-23 03:57:14 | 2026-10-23 03:57:14 | test-fashion-mnist-trainjob-h8djc-dataset-initializer-0-0-r97fj.18e10c778c3765c8 | - | Normal | AddedInterface | Add eth0 [10.132.0.46/23] from ovn-kubernetes | 2026-10-23 03:57:14 | 2026-10-23 03:57:14 | test-fashion-mnist-trainjob-h8djc-dataset-initializer-0-0-r97fj.18e10c778ff3a9b1 | spec.containers{dataset-initializer} | Normal | Pulled | Container image "quay.io/opendatahub/odh-th-torch-cpu-py312:odh-stable" already present on machine and can be accessed by the pod | 2026-10-23 03:57:14 | 2026-10-23 03:57:14 | test-fashion-mnist-trainjob-h8djc-dataset-initializer-0-0-r97fj.18e10c7792a8aac1 | spec.containers{dataset-initializer} | Normal | Created | Container created | 2026-10-23 03:57:14 | 2026-10-23 03:57:14 | test-fashion-mnist-trainjob-h8djc-dataset-initializer-0-0-r97fj.18e10c77930ed63e | spec.containers{dataset-initializer} | Normal | Started | Container started | 2026-10-23 03:57:12 | 2026-10-23 03:57:12 | test-fashion-mnist-trainjob-h8djc-dataset-initializer-0.18e10c772f5dcd42 | - | Normal | Suspended | Job suspended | 2026-10-23 03:57:13 | 2026-10-23 03:57:13 | test-fashion-mnist-trainjob-h8djc-dataset-initializer-0.18e10c7757a0cbf2 | - | Normal | SuccessfulCreate | Created pod: test-fashion-mnist-trainjob-h8djc-dataset-initializer-0-0-r97fj | 2026-10-23 03:57:13 | 2026-10-23 03:57:13 | test-fashion-mnist-trainjob-h8djc-dataset-initializer-0.18e10c7757a12eb2 | - | Normal | Resumed | Job resumed | 2026-10-23 03:57:35 | 2026-10-23 03:57:35 | test-fashion-mnist-trainjob-h8djc-dataset-initializer-0.18e10c7c7cda808c | - | Normal | Completed | Job completed | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | test-fashion-mnist-trainjob-h8djc-node-0-0-q6gxr.18e10c7c831c6377 | - | Normal | Scheduled | Successfully assigned test-ns-bglsz/test-fashion-mnist-trainjob-h8djc-node-0-0-q6gxr to ip-10-0-140-146.ec2.internal | 2026-10-23 03:57:36 | 2026-10-23 03:57:36 | test-fashion-mnist-trainjob-h8djc-node-0-0-q6gxr.18e10c7ca57ae790 | - | Normal | AddedInterface | Add eth0 [10.132.0.47/23] from ovn-kubernetes | 2026-10-23 03:57:36 | 2026-10-23 03:57:36 | test-fashion-mnist-trainjob-h8djc-node-0-0-q6gxr.18e10c7ca8e1c30f | spec.containers{node} | Normal | Pulled | Container image "quay.io/opendatahub/odh-th-torch-cpu-py312:odh-stable" already present on machine and can be accessed by the pod | 2026-10-23 03:57:36 | 2026-10-23 03:57:36 | test-fashion-mnist-trainjob-h8djc-node-0-0-q6gxr.18e10c7cabd3edcc | spec.containers{node} | Normal | Created | Container created | 2026-10-23 03:57:36 | 2026-10-23 03:57:36 | test-fashion-mnist-trainjob-h8djc-node-0-0-q6gxr.18e10c7cac4daeab | spec.containers{node} | Normal | Started | Container started | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | test-fashion-mnist-trainjob-h8djc-node-0-1-mq77k.18e10c7c8705da95 | - | Normal | Scheduled | Successfully assigned test-ns-bglsz/test-fashion-mnist-trainjob-h8djc-node-0-1-mq77k to ip-10-0-140-14.ec2.internal | 2026-10-23 03:57:36 | 2026-10-23 03:57:36 | test-fashion-mnist-trainjob-h8djc-node-0-1-mq77k.18e10c7cbb516844 | - | Normal | AddedInterface | Add eth0 [10.134.0.48/23] from ovn-kubernetes | 2026-10-23 03:57:36 | 2026-10-23 03:57:36 | test-fashion-mnist-trainjob-h8djc-node-0-1-mq77k.18e10c7cbed836fe | spec.containers{node} | Normal | Pulled | Container image "quay.io/opendatahub/odh-th-torch-cpu-py312:odh-stable" already present on machine and can be accessed by the pod | 2026-10-23 03:57:36 | 2026-10-23 03:57:36 | test-fashion-mnist-trainjob-h8djc-node-0-1-mq77k.18e10c7cc1dc76b6 | spec.containers{node} | Normal | Created | Container created | 2026-10-23 03:57:36 | 2026-10-23 03:57:36 | test-fashion-mnist-trainjob-h8djc-node-0-1-mq77k.18e10c7cc254b929 | spec.containers{node} | Normal | Started | Container started | 2026-10-23 03:57:35 | 2026-10-23 03:57:35 | test-fashion-mnist-trainjob-h8djc-node-0.18e10c7c82b43988 | - | Normal | SuccessfulCreate | Created pod: test-fashion-mnist-trainjob-h8djc-node-0-0-q6gxr | 2026-10-23 03:57:35 | 2026-10-23 03:57:35 | test-fashion-mnist-trainjob-h8djc-node-0.18e10c7c8692d256 | - | Normal | SuccessfulCreate | Created pod: test-fashion-mnist-trainjob-h8djc-node-0-1-mq77k | 2026-10-23 03:58:15 | 2026-10-23 03:58:15 | test-fashion-mnist-trainjob-h8djc-node-0.18e10c85d17e51e2 | - | Normal | Completed | Job completed | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | test-fashion-mnist-trainjob-h8djc.18e10c772cb0da53 | - | Normal | CreatedWorkload | Created Workload: test-ns-bglsz/trainjob-test-fashion-mnist-trainjob-h8djc-79d1c | 2026-10-23 03:57:12 | 2026-10-23 03:57:12 | test-fashion-mnist-trainjob-h8djc.18e10c773220c7bd | - | Normal | SuspendedJobs | jobset is suspended | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | test-fashion-mnist-trainjob-h8djc.18e10c7746b124e2 | - | Normal | Started | Admitted by clusterQueue cq-rtx77 | 2026-10-23 03:57:13 | 2026-10-23 03:57:13 | test-fashion-mnist-trainjob-h8djc.18e10c774d8a8cd8 | - | Normal | ResumeJobs | jobset is resumed | 2026-10-23 03:58:15 | 2026-10-23 03:58:15 | test-fashion-mnist-trainjob-h8djc.18e10c85d3a56cb1 | - | Normal | AllJobsCompleted | jobset completed successfully | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | test-fashion-mnist-trainjob-h8djc.18e10c85dd3b14ef | - | Normal | FinishedWorkload | Workload 'test-ns-bglsz/trainjob-test-fashion-mnist-trainjob-h8djc-79d1c' is declared finished | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | trainjob-test-fashion-mnist-trainjob-h8djc-79d1c.18e10c772f31d90a | - | Normal | QuotaReserved | Quota reserved in ClusterQueue cq-rtx77, wait time since queued was 1s | 0001-01-01 00:00:00 | 0001-01-01 00:00:00 | trainjob-test-fashion-mnist-trainjob-h8djc-79d1c.18e10c772f329265 | - | Normal | Admitted | Admitted by ClusterQueue cq-rtx77, wait time since reservation was 1s |