LAST_SEEN            | FIRST_SEEN           | NAME                                                         | SUBOBJECT              | TYPE     | REASON            | MESSAGE                                                                                                                                                  | 
0001-01-01 00:00:00  | 0001-01-01 00:00:00  | test-mpi-trainjob-2n7x6-launcher-0-0-sfkpb.18e43151788ecdde  | -                      | Normal   | Scheduled         | Successfully assigned test-ns-vvpz5/test-mpi-trainjob-2n7x6-launcher-0-0-sfkpb to ip-10-0-130-99.us-west-1.compute.internal                              | 
2026-11-02 09:46:18  | 2026-11-02 09:46:18  | test-mpi-trainjob-2n7x6-launcher-0-0-sfkpb.18e431519c5ac87e  | -                      | Normal   | AddedInterface    | Add eth0 [10.133.0.36/23] from ovn-kubernetes                                                                                                            | 
2026-11-02 09:46:18  | 2026-11-02 09:46:18  | test-mpi-trainjob-2n7x6-launcher-0-0-sfkpb.18e43151a3c757b3  | spec.containers{node}  | Normal   | Pulling           | Pulling image "quay.io/opendatahub/odh-th-torch-cpu-py312:odh-stable"                                                                                    | 
2026-11-02 09:47:14  | 2026-11-02 09:47:14  | test-mpi-trainjob-2n7x6-launcher-0-0-sfkpb.18e4315eb417d766  | spec.containers{node}  | Normal   | Pulled            | Successfully pulled image "quay.io/opendatahub/odh-th-torch-cpu-py312:odh-stable" in 56.108s (56.108s including waiting). Image size: 7472783034 bytes.  | 
2026-11-02 09:47:14  | 2026-11-02 09:47:14  | test-mpi-trainjob-2n7x6-launcher-0-0-sfkpb.18e4315eb8cd9ceb  | spec.containers{node}  | Normal   | Created           | Container created                                                                                                                                        | 
2026-11-02 09:47:14  | 2026-11-02 09:47:14  | test-mpi-trainjob-2n7x6-launcher-0-0-sfkpb.18e4315eb9fe0149  | spec.containers{node}  | Normal   | Started           | Container started                                                                                                                                        | 
2026-11-02 09:46:17  | 2026-11-02 09:46:17  | test-mpi-trainjob-2n7x6-launcher-0.18e4315178365a22          | -                      | Normal   | SuccessfulCreate  | Created pod: test-mpi-trainjob-2n7x6-launcher-0-0-sfkpb                                                                                                  | 
2026-11-02 09:47:25  | 2026-11-02 09:47:25  | test-mpi-trainjob-2n7x6-launcher-0.18e43161599b2ca7          | -                      | Normal   | Completed         | Job completed                                                                                                                                            | 
0001-01-01 00:00:00  | 0001-01-01 00:00:00  | test-mpi-trainjob-2n7x6-node-0-0-tbzmc.18e4314e603110e2      | -                      | Normal   | Scheduled         | Successfully assigned test-ns-vvpz5/test-mpi-trainjob-2n7x6-node-0-0-tbzmc to ip-10-0-135-30.us-west-1.compute.internal                                  | 
2026-11-02 09:46:04  | 2026-11-02 09:46:04  | test-mpi-trainjob-2n7x6-node-0-0-tbzmc.18e4314e843650b4      | -                      | Normal   | AddedInterface    | Add eth0 [10.134.0.45/23] from ovn-kubernetes                                                                                                            | 
2026-11-02 09:46:05  | 2026-11-02 09:46:05  | test-mpi-trainjob-2n7x6-node-0-0-tbzmc.18e4314e8bb45608      | spec.containers{node}  | Normal   | Pulled            | Container image "quay.io/opendatahub/odh-th-torch-cpu-py312:odh-stable" already present on machine and can be accessed by the pod                        | 
2026-11-02 09:46:05  | 2026-11-02 09:46:05  | test-mpi-trainjob-2n7x6-node-0-0-tbzmc.18e4314e8eb4824e      | spec.containers{node}  | Normal   | Created           | Container created                                                                                                                                        | 
2026-11-02 09:46:05  | 2026-11-02 09:46:05  | test-mpi-trainjob-2n7x6-node-0-0-tbzmc.18e4314e8f379c93      | spec.containers{node}  | Normal   | Started           | Container started                                                                                                                                        | 
2026-11-02 09:47:26  | 2026-11-02 09:47:26  | test-mpi-trainjob-2n7x6-node-0-0-tbzmc.18e431616fb72a8f      | spec.containers{node}  | Normal   | Killing           | Stopping container node                                                                                                                                  | 
2026-11-02 09:47:26  | 2026-11-02 09:47:26  | test-mpi-trainjob-2n7x6-node-0-0-tbzmc.18e43161771ab98f      | spec.containers{node}  | Warning  | Unhealthy         | Readiness probe failed: dial tcp 10.134.0.45:2222: connect: connection refused                                                                           | 
2026-11-02 09:46:04  | 2026-11-02 09:46:04  | test-mpi-trainjob-2n7x6-node-0.18e4314e5fc198b0              | -                      | Normal   | SuccessfulCreate  | Created pod: test-mpi-trainjob-2n7x6-node-0-0-tbzmc                                                                                                      | 
2026-11-02 09:47:25  | 2026-11-02 09:47:25  | test-mpi-trainjob-2n7x6.18e4316164f77e86                     | -                      | Normal   | AllJobsCompleted  | jobset completed successfully                                                                                                                            |