> Enter [BeforeSuite] TOP-LEVEL - /workspace/source/src_code/test/e2e/suite_test.go:43 @ 10/30/26 10:49:30.933 < Exit [BeforeSuite] TOP-LEVEL - /workspace/source/src_code/test/e2e/suite_test.go:43 @ 10/30/26 10:49:30.934 (1ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:49:30.934 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:49:31.083 (149ms) > Enter [It] should create TrainJob with PyTorch runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:84 @ 10/30/26 10:49:31.083 STEP: Create a TrainJob with torch-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:93 @ 10/30/26 10:49:31.083 END STEP: Create a TrainJob with torch-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:93 @ 10/30/26 10:49:31.219 (136ms) STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:98 @ 10/30/26 10:49:31.219 END STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:98 @ 10/30/26 10:49:31.769 (550ms) STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:116 @ 10/30/26 10:49:31.769 END STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:116 @ 10/30/26 10:54:22.075 (4m50.305s) < Exit [It] should create TrainJob with PyTorch runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:84 @ 10/30/26 10:54:22.075 (4m50.991s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:54:22.075 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:54:22.091 (16ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:54:22.092 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:54:22.125 (34ms) > Enter [It] should create TrainJob with DeepSpeed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:145 @ 10/30/26 10:54:22.125 [SKIPPED] MPI runtime image is not available In [It] at: /workspace/source/src_code/test/e2e/e2e_test.go:147 @ 10/30/26 10:54:22.125 < Exit [It] should create TrainJob with DeepSpeed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:145 @ 10/30/26 10:54:22.125 (0s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:54:22.125 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:54:22.144 (18ms) [FAILED] Expected success, but got an error: <*errors.StatusError | 0x1af16ad6bb80>: admission webhook "validator.trainjob.trainer.kubeflow.org" denied the request: spec.runtimeRef: Invalid value: {"name":"deepspeed-distributed","apiGroup":"trainer.kubeflow.org","kind":"ClusterTrainingRuntime"}: ClusterTrainingRuntime.trainer.kubeflow.org "deepspeed-distributed" not found: specified clusterTrainingRuntime must be created before the TrainJob is created { ErrStatus: { TypeMeta: {Kind: "", APIVersion: ""}, ListMeta: { SelfLink: "", ResourceVersion: "", Continue: "", RemainingItemCount: nil, ShardInfo: nil, }, Status: "Failure", Message: "admission webhook \"validator.trainjob.trainer.kubeflow.org\" denied the request: spec.runtimeRef: Invalid value: {\"name\":\"deepspeed-distributed\",\"apiGroup\":\"trainer.kubeflow.org\",\"kind\":\"ClusterTrainingRuntime\"}: ClusterTrainingRuntime.trainer.kubeflow.org \"deepspeed-distributed\" not found: specified clusterTrainingRuntime must be created before the TrainJob is created", Reason: "Forbidden", Details: nil, Code: 403, }, } In [It] at: /workspace/source/src_code/test/e2e/e2e_test.go:216 @ 10/30/26 10:54:22.294 > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:54:22.144 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:54:22.177 (33ms) > Enter [It] should create TrainJob with DeepSpeed runtime reference and OpenMPI 4.0 image - /workspace/source/src_code/test/e2e/e2e_test.go:203 @ 10/30/26 10:54:22.177 STEP: Create a TrainJob with deepspeed-distributed runtime reference and the OpenMPI 4.0 image - /workspace/source/src_code/test/e2e/e2e_test.go:215 @ 10/30/26 10:54:22.177 END STEP: Create a TrainJob with deepspeed-distributed runtime reference and the OpenMPI 4.0 image - /workspace/source/src_code/test/e2e/e2e_test.go:215 @ 10/30/26 10:54:22.294 (117ms) [FAILED] Expected success, but got an error: <*errors.StatusError | 0x1af16ad6bb80>: admission webhook "validator.trainjob.trainer.kubeflow.org" denied the request: spec.runtimeRef: Invalid value: {"name":"deepspeed-distributed","apiGroup":"trainer.kubeflow.org","kind":"ClusterTrainingRuntime"}: ClusterTrainingRuntime.trainer.kubeflow.org "deepspeed-distributed" not found: specified clusterTrainingRuntime must be created before the TrainJob is created { ErrStatus: { TypeMeta: {Kind: "", APIVersion: ""}, ListMeta: { SelfLink: "", ResourceVersion: "", Continue: "", RemainingItemCount: nil, ShardInfo: nil, }, Status: "Failure", Message: "admission webhook \"validator.trainjob.trainer.kubeflow.org\" denied the request: spec.runtimeRef: Invalid value: {\"name\":\"deepspeed-distributed\",\"apiGroup\":\"trainer.kubeflow.org\",\"kind\":\"ClusterTrainingRuntime\"}: ClusterTrainingRuntime.trainer.kubeflow.org \"deepspeed-distributed\" not found: specified clusterTrainingRuntime must be created before the TrainJob is created", Reason: "Forbidden", Details: nil, Code: 403, }, } In [It] at: /workspace/source/src_code/test/e2e/e2e_test.go:216 @ 10/30/26 10:54:22.294 < Exit [It] should create TrainJob with DeepSpeed runtime reference and OpenMPI 4.0 image - /workspace/source/src_code/test/e2e/e2e_test.go:203 @ 10/30/26 10:54:22.294 (117ms) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:54:22.294 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:54:22.311 (16ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:54:22.311 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:54:22.344 (33ms) > Enter [It] should create TrainJob with JAX runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:259 @ 10/30/26 10:54:22.344 STEP: Create a TrainJob with jax-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:265 @ 10/30/26 10:54:22.344 END STEP: Create a TrainJob with jax-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:265 @ 10/30/26 10:54:22.392 (48ms) STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:270 @ 10/30/26 10:54:22.392 END STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:270 @ 10/30/26 10:54:22.939 (547ms) STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:288 @ 10/30/26 10:54:22.939 END STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:288 @ 10/30/26 10:56:42.636 (2m19.697s) < Exit [It] should create TrainJob with JAX runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:259 @ 10/30/26 10:56:42.636 (2m20.292s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:56:42.636 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:56:42.654 (18ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:56:42.655 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:56:42.689 (34ms) > Enter [It] should create TrainJob with XGBoost runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:317 @ 10/30/26 10:56:42.689 STEP: Create a TrainJob with xgboost-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:323 @ 10/30/26 10:56:42.69 END STEP: Create a TrainJob with xgboost-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:323 @ 10/30/26 10:56:42.805 (115ms) STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:328 @ 10/30/26 10:56:42.805 END STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:328 @ 10/30/26 10:56:43.353 (549ms) STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:346 @ 10/30/26 10:56:43.353 END STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:346 @ 10/30/26 10:57:28.772 (45.419s) < Exit [It] should create TrainJob with XGBoost runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:317 @ 10/30/26 10:57:28.772 (46.082s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:57:28.772 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:57:28.789 (17ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:57:28.789 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:57:28.822 (33ms) > Enter [It] should create TrainJob with Flux runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:374 @ 10/30/26 10:57:28.822 STEP: Create a TrainJob with flux-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:393 @ 10/30/26 10:57:28.822 END STEP: Create a TrainJob with flux-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:393 @ 10/30/26 10:57:28.87 (49ms) STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:397 @ 10/30/26 10:57:28.87 END STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:397 @ 10/30/26 10:57:29.42 (550ms) STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:414 @ 10/30/26 10:57:29.42 END STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:414 @ 10/30/26 10:57:56.804 (27.384s) < Exit [It] should create TrainJob with Flux runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:374 @ 10/30/26 10:57:56.804 (27.982s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:57:56.804 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:57:56.821 (17ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:57:56.821 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:57:56.854 (33ms) > Enter [It] should preserve user-provided manager fields - /workspace/source/src_code/test/e2e/e2e_test.go:442 @ 10/30/26 10:57:56.854 STEP: Create a TrainJob with RuntimePatches - /workspace/source/src_code/test/e2e/e2e_test.go:494 @ 10/30/26 10:57:56.854 END STEP: Create a TrainJob with RuntimePatches - /workspace/source/src_code/test/e2e/e2e_test.go:494 @ 10/30/26 10:57:56.904 (50ms) STEP: Verify user-provided Time is preserved for test.io/manager-one and Time is set for kueue.k8s.io/manager - /workspace/source/src_code/test/e2e/e2e_test.go:498 @ 10/30/26 10:57:56.904 END STEP: Verify user-provided Time is preserved for test.io/manager-one and Time is set for kueue.k8s.io/manager - /workspace/source/src_code/test/e2e/e2e_test.go:498 @ 10/30/26 10:57:56.919 (15ms) < Exit [It] should preserve user-provided manager fields - /workspace/source/src_code/test/e2e/e2e_test.go:442 @ 10/30/26 10:57:56.919 (65ms) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:57:56.919 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:57:56.935 (16ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:57:56.935 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:57:56.967 (32ms) > Enter [It] should update Time only for the changed patch on update - /workspace/source/src_code/test/e2e/e2e_test.go:512 @ 10/30/26 10:57:56.967 STEP: Create a suspended TrainJob with two RuntimePatches - /workspace/source/src_code/test/e2e/e2e_test.go:566 @ 10/30/26 10:57:56.967 END STEP: Create a suspended TrainJob with two RuntimePatches - /workspace/source/src_code/test/e2e/e2e_test.go:566 @ 10/30/26 10:57:57.014 (48ms) STEP: Record the original Time values and verify user-provided Time is preserved - /workspace/source/src_code/test/e2e/e2e_test.go:571 @ 10/30/26 10:57:57.014 END STEP: Record the original Time values and verify user-provided Time is preserved - /workspace/source/src_code/test/e2e/e2e_test.go:571 @ 10/30/26 10:57:57.03 (15ms) STEP: Update the second RuntimePatch and add two new patches - /workspace/source/src_code/test/e2e/e2e_test.go:588 @ 10/30/26 10:57:58.03 END STEP: Update the second RuntimePatch and add two new patches - /workspace/source/src_code/test/e2e/e2e_test.go:588 @ 10/30/26 10:57:58.105 (75ms) STEP: Verify Time behaviour for all patches after update - /workspace/source/src_code/test/e2e/e2e_test.go:607 @ 10/30/26 10:57:58.105 END STEP: Verify Time behaviour for all patches after update - /workspace/source/src_code/test/e2e/e2e_test.go:607 @ 10/30/26 10:57:58.121 (16ms) < Exit [It] should update Time only for the changed patch on update - /workspace/source/src_code/test/e2e/e2e_test.go:512 @ 10/30/26 10:57:58.121 (1.155s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:57:58.121 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:57:58.138 (17ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:57:58.139 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:57:58.17 (32ms) > Enter [It] should not be reconciled by the built-in controller - /workspace/source/src_code/test/e2e/e2e_test.go:629 @ 10/30/26 10:57:58.17 STEP: Create a TrainJob managed by MultiKueue - /workspace/source/src_code/test/e2e/e2e_test.go:636 @ 10/30/26 10:57:58.17 END STEP: Create a TrainJob managed by MultiKueue - /workspace/source/src_code/test/e2e/e2e_test.go:636 @ 10/30/26 10:57:58.22 (50ms) STEP: Ensuring the built-in controller neither creates a JobSet nor sets any status - /workspace/source/src_code/test/e2e/e2e_test.go:640 @ 10/30/26 10:57:58.22 END STEP: Ensuring the built-in controller neither creates a JobSet nor sets any status - /workspace/source/src_code/test/e2e/e2e_test.go:640 @ 10/30/26 10:58:13.221 (15.001s) < Exit [It] should not be reconciled by the built-in controller - /workspace/source/src_code/test/e2e/e2e_test.go:629 @ 10/30/26 10:58:13.221 (15.051s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:58:13.221 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:58:13.239 (19ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:58:13.24 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:58:13.271 (31ms) > Enter [It] should fail the TrainJob with DeadlineExceeded when active timeout expires - /workspace/source/src_code/test/e2e/e2e_test.go:652 @ 10/30/26 10:58:13.271 STEP: Waiting for TrainJob to fail due to deadline - /workspace/source/src_code/test/e2e/e2e_test.go:667 @ 10/30/26 10:58:13.326 STEP: Ensuring the underlying JobSet is deleted - /workspace/source/src_code/test/e2e/e2e_test.go:674 @ 10/30/26 10:58:23.179 < Exit [It] should fail the TrainJob with DeadlineExceeded when active timeout expires - /workspace/source/src_code/test/e2e/e2e_test.go:652 @ 10/30/26 10:58:23.194 (9.923s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:58:23.194 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:58:23.211 (17ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:58:23.211 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:58:23.244 (33ms) > Enter [It] TrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:905 @ 10/30/26 10:58:23.244 STEP: Creating a TrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:797 @ 10/30/26 10:58:23.244 END STEP: Creating a TrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:797 @ 10/30/26 10:58:23.322 (78ms) STEP: Creating a TrainJob that references the TrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:810 @ 10/30/26 10:58:23.322 END STEP: Creating a TrainJob that references the TrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:810 @ 10/30/26 10:58:23.367 (45ms) STEP: Waiting for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:814 @ 10/30/26 10:58:23.367 END STEP: Waiting for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:814 @ 10/30/26 10:58:23.647 (280ms) STEP: Pausing the TrainJob - /workspace/source/src_code/test/e2e/e2e_test.go:825 @ 10/30/26 10:58:23.647 END STEP: Pausing the TrainJob - /workspace/source/src_code/test/e2e/e2e_test.go:825 @ 10/30/26 10:58:23.712 (65ms) STEP: Waiting for TrainJob to be suspended - /workspace/source/src_code/test/e2e/e2e_test.go:834 @ 10/30/26 10:58:23.712 END STEP: Waiting for TrainJob to be suspended - /workspace/source/src_code/test/e2e/e2e_test.go:834 @ 10/30/26 10:58:23.992 (280ms) STEP: Updating the runtime with a new container image - /workspace/source/src_code/test/e2e/e2e_test.go:847 @ 10/30/26 10:58:23.992 END STEP: Updating the runtime with a new container image - /workspace/source/src_code/test/e2e/e2e_test.go:847 @ 10/30/26 10:58:24.041 (49ms) STEP: Restarting the TrainJob by unpausing it - /workspace/source/src_code/test/e2e/e2e_test.go:868 @ 10/30/26 10:58:24.041 END STEP: Restarting the TrainJob by unpausing it - /workspace/source/src_code/test/e2e/e2e_test.go:868 @ 10/30/26 10:58:24.132 (91ms) STEP: Verifying the TrainJob jobs become active after resuming - /workspace/source/src_code/test/e2e/e2e_test.go:879 @ 10/30/26 10:58:24.132 END STEP: Verifying the TrainJob jobs become active after resuming - /workspace/source/src_code/test/e2e/e2e_test.go:879 @ 10/30/26 10:58:24.414 (282ms) STEP: Verifying the Runtime snapshot is garbage collected - /workspace/source/src_code/test/e2e/e2e_test.go:891 @ 10/30/26 10:58:24.414 END STEP: Verifying the Runtime snapshot is garbage collected - /workspace/source/src_code/test/e2e/e2e_test.go:891 @ 10/30/26 10:58:24.463 (49ms) < Exit [It] TrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:905 @ 10/30/26 10:58:24.463 (1.219s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:58:24.463 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:58:24.481 (18ms) > Enter [DeferCleanup (Each)] a suspended TrainJob should use the original runtime configuration and not pick up the new configuration - /workspace/source/src_code/test/e2e/e2e_test.go:801 @ 10/30/26 10:58:24.481 < Exit [DeferCleanup (Each)] a suspended TrainJob should use the original runtime configuration and not pick up the new configuration - /workspace/source/src_code/test/e2e/e2e_test.go:801 @ 10/30/26 10:58:24.503 (22ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:58:24.503 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:61 @ 10/30/26 10:58:24.544 (41ms) > Enter [It] ClusterTrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:917 @ 10/30/26 10:58:24.544 STEP: Creating a ClusterTrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:797 @ 10/30/26 10:58:24.544 END STEP: Creating a ClusterTrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:797 @ 10/30/26 10:58:24.645 (101ms) STEP: Creating a TrainJob that references the ClusterTrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:810 @ 10/30/26 10:58:24.645 END STEP: Creating a TrainJob that references the ClusterTrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:810 @ 10/30/26 10:58:24.701 (57ms) STEP: Waiting for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:814 @ 10/30/26 10:58:24.701 END STEP: Waiting for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:814 @ 10/30/26 10:58:24.985 (284ms) STEP: Pausing the TrainJob - /workspace/source/src_code/test/e2e/e2e_test.go:825 @ 10/30/26 10:58:24.985 END STEP: Pausing the TrainJob - /workspace/source/src_code/test/e2e/e2e_test.go:825 @ 10/30/26 10:58:25.047 (62ms) STEP: Waiting for TrainJob to be suspended - /workspace/source/src_code/test/e2e/e2e_test.go:834 @ 10/30/26 10:58:25.047 END STEP: Waiting for TrainJob to be suspended - /workspace/source/src_code/test/e2e/e2e_test.go:834 @ 10/30/26 10:58:25.327 (280ms) STEP: Updating the runtime with a new container image - /workspace/source/src_code/test/e2e/e2e_test.go:847 @ 10/30/26 10:58:25.327 END STEP: Updating the runtime with a new container image - /workspace/source/src_code/test/e2e/e2e_test.go:847 @ 10/30/26 10:58:25.378 (51ms) STEP: Restarting the TrainJob by unpausing it - /workspace/source/src_code/test/e2e/e2e_test.go:868 @ 10/30/26 10:58:25.378 END STEP: Restarting the TrainJob by unpausing it - /workspace/source/src_code/test/e2e/e2e_test.go:868 @ 10/30/26 10:58:25.44 (63ms) STEP: Verifying the TrainJob jobs become active after resuming - /workspace/source/src_code/test/e2e/e2e_test.go:879 @ 10/30/26 10:58:25.44 END STEP: Verifying the TrainJob jobs become active after resuming - /workspace/source/src_code/test/e2e/e2e_test.go:879 @ 10/30/26 10:58:25.721 (280ms) STEP: Verifying the Runtime snapshot is garbage collected - /workspace/source/src_code/test/e2e/e2e_test.go:891 @ 10/30/26 10:58:25.721 END STEP: Verifying the Runtime snapshot is garbage collected - /workspace/source/src_code/test/e2e/e2e_test.go:891 @ 10/30/26 10:58:25.771 (50ms) < Exit [It] ClusterTrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:917 @ 10/30/26 10:58:25.771 (1.227s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:58:25.771 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:76 @ 10/30/26 10:58:25.791 (20ms) > Enter [DeferCleanup (Each)] a suspended TrainJob should use the original runtime configuration and not pick up the new configuration - /workspace/source/src_code/test/e2e/e2e_test.go:801 @ 10/30/26 10:58:25.791 < Exit [DeferCleanup (Each)] a suspended TrainJob should use the original runtime configuration and not pick up the new configuration - /workspace/source/src_code/test/e2e/e2e_test.go:801 @ 10/30/26 10:58:25.81 (19ms)