> Enter [BeforeSuite] TOP-LEVEL - /workspace/source/src_code/test/e2e/suite_test.go:43 @ 10/30/26 18:49:05.979 < Exit [BeforeSuite] TOP-LEVEL - /workspace/source/src_code/test/e2e/suite_test.go:43 @ 10/30/26 18:49:05.98 (1ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:49:05.98 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:49:06.141 (161ms) > Enter [It] should create TrainJob with PyTorch runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:84 @ 10/30/26 18:49:06.141 STEP: Create a TrainJob with torch-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:93 @ 10/30/26 18:49:06.141 END STEP: Create a TrainJob with torch-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:93 @ 10/30/26 18:49:06.285 (144ms) STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:98 @ 10/30/26 18:49:06.285 END STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:98 @ 10/30/26 18:49:06.837 (552ms) STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:116 @ 10/30/26 18:49:06.837 END STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:116 @ 10/30/26 18:54:01.872 (4m55.035s) < Exit [It] should create TrainJob with PyTorch runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:84 @ 10/30/26 18:54:01.872 (4m55.731s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:54:01.872 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:54:01.891 (19ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:54:01.892 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:54:01.928 (36ms) > Enter [It] should create TrainJob with DeepSpeed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:145 @ 10/30/26 18:54:01.928 [SKIPPED] MPI runtime image is not available In [It] at: /workspace/source/src_code/test/e2e/e2e_test.go:147 @ 10/30/26 18:54:01.928 < Exit [It] should create TrainJob with DeepSpeed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:145 @ 10/30/26 18:54:01.928 (0s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:54:01.928 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:54:01.99 (61ms) [FAILED] Expected success, but got an error: <*errors.StatusError | 0x811a8f30be0>: admission webhook "validator.trainjob.trainer.kubeflow.org" denied the request: spec.runtimeRef: Invalid value: {"name":"deepspeed-distributed","apiGroup":"trainer.kubeflow.org","kind":"ClusterTrainingRuntime"}: ClusterTrainingRuntime.trainer.kubeflow.org "deepspeed-distributed" not found: specified clusterTrainingRuntime must be created before the TrainJob is created { ErrStatus: { TypeMeta: {Kind: "", APIVersion: ""}, ListMeta: { SelfLink: "", ResourceVersion: "", Continue: "", RemainingItemCount: nil, ShardInfo: nil, }, Status: "Failure", Message: "admission webhook \"validator.trainjob.trainer.kubeflow.org\" denied the request: spec.runtimeRef: Invalid value: {\"name\":\"deepspeed-distributed\",\"apiGroup\":\"trainer.kubeflow.org\",\"kind\":\"ClusterTrainingRuntime\"}: ClusterTrainingRuntime.trainer.kubeflow.org \"deepspeed-distributed\" not found: specified clusterTrainingRuntime must be created before the TrainJob is created", Reason: "Forbidden", Details: nil, Code: 403, }, } In [It] at: /workspace/source/src_code/test/e2e/e2e_test.go:216 @ 10/30/26 18:54:02.275 > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:54:01.99 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:54:02.05 (60ms) > Enter [It] should create TrainJob with DeepSpeed runtime reference and OpenMPI 4.0 image - /workspace/source/src_code/test/e2e/e2e_test.go:203 @ 10/30/26 18:54:02.05 STEP: Create a TrainJob with deepspeed-distributed runtime reference and the OpenMPI 4.0 image - /workspace/source/src_code/test/e2e/e2e_test.go:215 @ 10/30/26 18:54:02.05 END STEP: Create a TrainJob with deepspeed-distributed runtime reference and the OpenMPI 4.0 image - /workspace/source/src_code/test/e2e/e2e_test.go:215 @ 10/30/26 18:54:02.275 (225ms) [FAILED] Expected success, but got an error: <*errors.StatusError | 0x811a8f30be0>: admission webhook "validator.trainjob.trainer.kubeflow.org" denied the request: spec.runtimeRef: Invalid value: {"name":"deepspeed-distributed","apiGroup":"trainer.kubeflow.org","kind":"ClusterTrainingRuntime"}: ClusterTrainingRuntime.trainer.kubeflow.org "deepspeed-distributed" not found: specified clusterTrainingRuntime must be created before the TrainJob is created { ErrStatus: { TypeMeta: {Kind: "", APIVersion: ""}, ListMeta: { SelfLink: "", ResourceVersion: "", Continue: "", RemainingItemCount: nil, ShardInfo: nil, }, Status: "Failure", Message: "admission webhook \"validator.trainjob.trainer.kubeflow.org\" denied the request: spec.runtimeRef: Invalid value: {\"name\":\"deepspeed-distributed\",\"apiGroup\":\"trainer.kubeflow.org\",\"kind\":\"ClusterTrainingRuntime\"}: ClusterTrainingRuntime.trainer.kubeflow.org \"deepspeed-distributed\" not found: specified clusterTrainingRuntime must be created before the TrainJob is created", Reason: "Forbidden", Details: nil, Code: 403, }, } In [It] at: /workspace/source/src_code/test/e2e/e2e_test.go:216 @ 10/30/26 18:54:02.275 < Exit [It] should create TrainJob with DeepSpeed runtime reference and OpenMPI 4.0 image - /workspace/source/src_code/test/e2e/e2e_test.go:203 @ 10/30/26 18:54:02.275 (225ms) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:54:02.275 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:54:02.294 (19ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:54:02.294 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:54:02.329 (35ms) > Enter [It] should create TrainJob with JAX runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:259 @ 10/30/26 18:54:02.329 STEP: Create a TrainJob with jax-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:265 @ 10/30/26 18:54:02.329 END STEP: Create a TrainJob with jax-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:265 @ 10/30/26 18:54:02.374 (45ms) STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:270 @ 10/30/26 18:54:02.374 END STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:270 @ 10/30/26 18:54:02.925 (551ms) STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:288 @ 10/30/26 18:54:02.925 END STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:288 @ 10/30/26 18:56:31.655 (2m28.73s) < Exit [It] should create TrainJob with JAX runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:259 @ 10/30/26 18:56:31.655 (2m29.325s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:56:31.655 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:56:31.673 (18ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:56:31.673 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:56:31.709 (35ms) > Enter [It] should create TrainJob with XGBoost runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:317 @ 10/30/26 18:56:31.709 STEP: Create a TrainJob with xgboost-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:323 @ 10/30/26 18:56:31.709 END STEP: Create a TrainJob with xgboost-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:323 @ 10/30/26 18:56:31.852 (143ms) STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:328 @ 10/30/26 18:56:31.852 END STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:328 @ 10/30/26 18:56:32.401 (549ms) STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:346 @ 10/30/26 18:56:32.401 END STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:346 @ 10/30/26 18:57:17.795 (45.394s) < Exit [It] should create TrainJob with XGBoost runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:317 @ 10/30/26 18:57:17.795 (46.087s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:57:17.795 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:57:17.816 (20ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:57:17.816 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:57:17.85 (34ms) > Enter [It] should create TrainJob with Flux runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:374 @ 10/30/26 18:57:17.85 STEP: Create a TrainJob with flux-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:393 @ 10/30/26 18:57:17.85 END STEP: Create a TrainJob with flux-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:393 @ 10/30/26 18:57:17.903 (52ms) STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:397 @ 10/30/26 18:57:17.903 END STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:397 @ 10/30/26 18:57:18.454 (551ms) STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:414 @ 10/30/26 18:57:18.454 END STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:414 @ 10/30/26 18:57:45.442 (26.988s) < Exit [It] should create TrainJob with Flux runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:374 @ 10/30/26 18:57:45.442 (27.591s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:57:45.442 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:57:45.46 (19ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:57:45.46 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:57:45.495 (34ms) > Enter [It] should preserve user-provided manager fields - /workspace/source/src_code/test/e2e/e2e_test.go:442 @ 10/30/26 18:57:45.495 STEP: Create a TrainJob with RuntimePatches - /workspace/source/src_code/test/e2e/e2e_test.go:494 @ 10/30/26 18:57:45.495 END STEP: Create a TrainJob with RuntimePatches - /workspace/source/src_code/test/e2e/e2e_test.go:494 @ 10/30/26 18:57:45.554 (59ms) STEP: Verify user-provided Time is preserved for test.io/manager-one and Time is set for kueue.k8s.io/manager - /workspace/source/src_code/test/e2e/e2e_test.go:498 @ 10/30/26 18:57:45.554 END STEP: Verify user-provided Time is preserved for test.io/manager-one and Time is set for kueue.k8s.io/manager - /workspace/source/src_code/test/e2e/e2e_test.go:498 @ 10/30/26 18:57:45.57 (16ms) < Exit [It] should preserve user-provided manager fields - /workspace/source/src_code/test/e2e/e2e_test.go:442 @ 10/30/26 18:57:45.57 (75ms) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:57:45.57 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:57:45.588 (17ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:57:45.588 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:57:45.622 (34ms) > Enter [It] should update Time only for the changed patch on update - /workspace/source/src_code/test/e2e/e2e_test.go:512 @ 10/30/26 18:57:45.622 STEP: Create a suspended TrainJob with two RuntimePatches - /workspace/source/src_code/test/e2e/e2e_test.go:566 @ 10/30/26 18:57:45.622 END STEP: Create a suspended TrainJob with two RuntimePatches - /workspace/source/src_code/test/e2e/e2e_test.go:566 @ 10/30/26 18:57:45.674 (51ms) STEP: Record the original Time values and verify user-provided Time is preserved - /workspace/source/src_code/test/e2e/e2e_test.go:571 @ 10/30/26 18:57:45.674 END STEP: Record the original Time values and verify user-provided Time is preserved - /workspace/source/src_code/test/e2e/e2e_test.go:571 @ 10/30/26 18:57:45.69 (17ms) STEP: Update the second RuntimePatch and add two new patches - /workspace/source/src_code/test/e2e/e2e_test.go:588 @ 10/30/26 18:57:46.691 END STEP: Update the second RuntimePatch and add two new patches - /workspace/source/src_code/test/e2e/e2e_test.go:588 @ 10/30/26 18:57:46.765 (75ms) STEP: Verify Time behaviour for all patches after update - /workspace/source/src_code/test/e2e/e2e_test.go:607 @ 10/30/26 18:57:46.765 END STEP: Verify Time behaviour for all patches after update - /workspace/source/src_code/test/e2e/e2e_test.go:607 @ 10/30/26 18:57:46.782 (17ms) < Exit [It] should update Time only for the changed patch on update - /workspace/source/src_code/test/e2e/e2e_test.go:512 @ 10/30/26 18:57:46.782 (1.16s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:57:46.782 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:57:46.802 (20ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:57:46.802 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:57:46.837 (36ms) > Enter [It] should not be reconciled by the built-in controller - /workspace/source/src_code/test/e2e/e2e_test.go:629 @ 10/30/26 18:57:46.837 STEP: Create a TrainJob managed by MultiKueue - /workspace/source/src_code/test/e2e/e2e_test.go:636 @ 10/30/26 18:57:46.838 END STEP: Create a TrainJob managed by MultiKueue - /workspace/source/src_code/test/e2e/e2e_test.go:636 @ 10/30/26 18:57:46.888 (51ms) STEP: Ensuring the built-in controller neither creates a JobSet nor sets any status - /workspace/source/src_code/test/e2e/e2e_test.go:640 @ 10/30/26 18:57:46.888 END STEP: Ensuring the built-in controller neither creates a JobSet nor sets any status - /workspace/source/src_code/test/e2e/e2e_test.go:640 @ 10/30/26 18:58:01.902 (15.014s) < Exit [It] should not be reconciled by the built-in controller - /workspace/source/src_code/test/e2e/e2e_test.go:629 @ 10/30/26 18:58:01.902 (15.065s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:58:01.902 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:58:01.927 (25ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:58:01.927 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:58:01.964 (37ms) > Enter [It] should fail the TrainJob with DeadlineExceeded when active timeout expires - /workspace/source/src_code/test/e2e/e2e_test.go:652 @ 10/30/26 18:58:01.964 STEP: Waiting for TrainJob to fail due to deadline - /workspace/source/src_code/test/e2e/e2e_test.go:667 @ 10/30/26 18:58:02.013 STEP: Ensuring the underlying JobSet is deleted - /workspace/source/src_code/test/e2e/e2e_test.go:674 @ 10/30/26 18:58:11.108 < Exit [It] should fail the TrainJob with DeadlineExceeded when active timeout expires - /workspace/source/src_code/test/e2e/e2e_test.go:652 @ 10/30/26 18:58:11.392 (9.428s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:58:11.392 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:58:11.412 (21ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:58:11.413 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:58:11.452 (39ms) > Enter [It] TrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:905 @ 10/30/26 18:58:11.452 STEP: Creating a TrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:797 @ 10/30/26 18:58:11.452 END STEP: Creating a TrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:797 @ 10/30/26 18:58:11.541 (89ms) STEP: Creating a TrainJob that references the TrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:810 @ 10/30/26 18:58:11.541 END STEP: Creating a TrainJob that references the TrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:810 @ 10/30/26 18:58:11.59 (49ms) STEP: Waiting for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:814 @ 10/30/26 18:58:11.59 END STEP: Waiting for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:814 @ 10/30/26 18:58:11.872 (283ms) STEP: Pausing the TrainJob - /workspace/source/src_code/test/e2e/e2e_test.go:825 @ 10/30/26 18:58:11.873 END STEP: Pausing the TrainJob - /workspace/source/src_code/test/e2e/e2e_test.go:825 @ 10/30/26 18:58:11.939 (66ms) STEP: Waiting for TrainJob to be suspended - /workspace/source/src_code/test/e2e/e2e_test.go:834 @ 10/30/26 18:58:11.939 END STEP: Waiting for TrainJob to be suspended - /workspace/source/src_code/test/e2e/e2e_test.go:834 @ 10/30/26 18:58:12.225 (286ms) STEP: Updating the runtime with a new container image - /workspace/source/src_code/test/e2e/e2e_test.go:847 @ 10/30/26 18:58:12.225 END STEP: Updating the runtime with a new container image - /workspace/source/src_code/test/e2e/e2e_test.go:847 @ 10/30/26 18:58:12.277 (52ms) STEP: Restarting the TrainJob by unpausing it - /workspace/source/src_code/test/e2e/e2e_test.go:868 @ 10/30/26 18:58:12.277 END STEP: Restarting the TrainJob by unpausing it - /workspace/source/src_code/test/e2e/e2e_test.go:868 @ 10/30/26 18:58:12.347 (69ms) STEP: Verifying the TrainJob jobs become active after resuming - /workspace/source/src_code/test/e2e/e2e_test.go:879 @ 10/30/26 18:58:12.347 END STEP: Verifying the TrainJob jobs become active after resuming - /workspace/source/src_code/test/e2e/e2e_test.go:879 @ 10/30/26 18:58:12.63 (283ms) STEP: Verifying the Runtime snapshot is garbage collected - /workspace/source/src_code/test/e2e/e2e_test.go:891 @ 10/30/26 18:58:12.63 END STEP: Verifying the Runtime snapshot is garbage collected - /workspace/source/src_code/test/e2e/e2e_test.go:891 @ 10/30/26 18:58:12.681 (51ms) < Exit [It] TrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:905 @ 10/30/26 18:58:12.681 (1.229s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:58:12.681 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:58:12.705 (24ms) > Enter [DeferCleanup (Each)] a suspended TrainJob should use the original runtime configuration and not pick up the new configuration - /workspace/source/src_code/test/e2e/e2e_test.go:801 @ 10/30/26 18:58:12.705 < Exit [DeferCleanup (Each)] a suspended TrainJob should use the original runtime configuration and not pick up the new configuration - /workspace/source/src_code/test/e2e/e2e_test.go:801 @ 10/30/26 18:58:12.723 (19ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:58:12.723 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 18:58:12.763 (40ms) > Enter [It] ClusterTrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:917 @ 10/30/26 18:58:12.763 STEP: Creating a ClusterTrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:797 @ 10/30/26 18:58:12.764 END STEP: Creating a ClusterTrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:797 @ 10/30/26 18:58:12.846 (83ms) STEP: Creating a TrainJob that references the ClusterTrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:810 @ 10/30/26 18:58:12.846 END STEP: Creating a TrainJob that references the ClusterTrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:810 @ 10/30/26 18:58:12.901 (55ms) STEP: Waiting for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:814 @ 10/30/26 18:58:12.901 END STEP: Waiting for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:814 @ 10/30/26 18:58:13.186 (285ms) STEP: Pausing the TrainJob - /workspace/source/src_code/test/e2e/e2e_test.go:825 @ 10/30/26 18:58:13.186 END STEP: Pausing the TrainJob - /workspace/source/src_code/test/e2e/e2e_test.go:825 @ 10/30/26 18:58:13.253 (67ms) STEP: Waiting for TrainJob to be suspended - /workspace/source/src_code/test/e2e/e2e_test.go:834 @ 10/30/26 18:58:13.253 END STEP: Waiting for TrainJob to be suspended - /workspace/source/src_code/test/e2e/e2e_test.go:834 @ 10/30/26 18:58:13.536 (283ms) STEP: Updating the runtime with a new container image - /workspace/source/src_code/test/e2e/e2e_test.go:847 @ 10/30/26 18:58:13.536 END STEP: Updating the runtime with a new container image - /workspace/source/src_code/test/e2e/e2e_test.go:847 @ 10/30/26 18:58:13.591 (55ms) STEP: Restarting the TrainJob by unpausing it - /workspace/source/src_code/test/e2e/e2e_test.go:868 @ 10/30/26 18:58:13.591 END STEP: Restarting the TrainJob by unpausing it - /workspace/source/src_code/test/e2e/e2e_test.go:868 @ 10/30/26 18:58:13.66 (69ms) STEP: Verifying the TrainJob jobs become active after resuming - /workspace/source/src_code/test/e2e/e2e_test.go:879 @ 10/30/26 18:58:13.66 END STEP: Verifying the TrainJob jobs become active after resuming - /workspace/source/src_code/test/e2e/e2e_test.go:879 @ 10/30/26 18:58:13.943 (282ms) STEP: Verifying the Runtime snapshot is garbage collected - /workspace/source/src_code/test/e2e/e2e_test.go:891 @ 10/30/26 18:58:13.943 END STEP: Verifying the Runtime snapshot is garbage collected - /workspace/source/src_code/test/e2e/e2e_test.go:891 @ 10/30/26 18:58:13.993 (51ms) < Exit [It] ClusterTrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:917 @ 10/30/26 18:58:13.993 (1.23s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:58:13.993 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 18:58:14.013 (20ms) > Enter [DeferCleanup (Each)] a suspended TrainJob should use the original runtime configuration and not pick up the new configuration - /workspace/source/src_code/test/e2e/e2e_test.go:801 @ 10/30/26 18:58:14.013 < Exit [DeferCleanup (Each)] a suspended TrainJob should use the original runtime configuration and not pick up the new configuration - /workspace/source/src_code/test/e2e/e2e_test.go:801 @ 10/30/26 18:58:14.033 (19ms)