> Enter [BeforeSuite] TOP-LEVEL - /workspace/source/src_code/test/e2e/suite_test.go:43 @ 09/19/26 02:59:18.973 < Exit [BeforeSuite] TOP-LEVEL - /workspace/source/src_code/test/e2e/suite_test.go:43 @ 09/19/26 02:59:18.974 (1ms) [FAILED] Timed out after 600.001s. The function passed to Eventually failed at /workspace/source/src_code/test/e2e/e2e_test.go:112 with: Expected object to be comparable, diff: []v1.Condition( - nil, + { + { + Type: "Complete", + Status: "True", + Reason: "AllJobsCompleted", + Message: "jobset completed successfully", + }, + }, ) In [It] at: /workspace/source/src_code/test/e2e/e2e_test.go:130 @ 09/19/26 03:09:19.324 > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:56 @ 09/19/26 02:59:18.974 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:56 @ 09/19/26 02:59:19.021 (47ms) > Enter [It] should create TrainJob with PyTorch runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:79 @ 09/19/26 02:59:19.021 STEP: Create a TrainJob with torch-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:85 @ 09/19/26 02:59:19.021 END STEP: Create a TrainJob with torch-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:85 @ 09/19/26 02:59:19.066 (45ms) STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:90 @ 09/19/26 02:59:19.066 END STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:90 @ 09/19/26 02:59:19.323 (257ms) STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:108 @ 09/19/26 02:59:19.323 END STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:108 @ 09/19/26 03:09:19.324 (10m0.001s) [FAILED] Timed out after 600.001s. The function passed to Eventually failed at /workspace/source/src_code/test/e2e/e2e_test.go:112 with: Expected object to be comparable, diff: []v1.Condition( - nil, + { + { + Type: "Complete", + Status: "True", + Reason: "AllJobsCompleted", + Message: "jobset completed successfully", + }, + }, ) In [It] at: /workspace/source/src_code/test/e2e/e2e_test.go:130 @ 09/19/26 03:09:19.324 < Exit [It] should create TrainJob with PyTorch runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:79 @ 09/19/26 03:09:19.324 (10m0.303s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:71 @ 09/19/26 03:09:19.324 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:71 @ 09/19/26 03:09:19.329 (5ms) [FAILED] Expected success, but got an error: <*errors.StatusError | 0x2ee407e9ba40>: admission webhook "validator.trainjob.trainer.kubeflow.org" denied the request: spec.RuntimeRef: Invalid value: {"name":"deepspeed-distributed","apiGroup":"trainer.kubeflow.org","kind":"ClusterTrainingRuntime"}: ClusterTrainingRuntime.trainer.kubeflow.org "deepspeed-distributed" not found: specified clusterTrainingRuntime must be created before the TrainJob is created { ErrStatus: { TypeMeta: {Kind: "", APIVersion: ""}, ListMeta: { SelfLink: "", ResourceVersion: "", Continue: "", RemainingItemCount: nil, ShardInfo: nil, }, Status: "Failure", Message: "admission webhook \"validator.trainjob.trainer.kubeflow.org\" denied the request: spec.RuntimeRef: Invalid value: {\"name\":\"deepspeed-distributed\",\"apiGroup\":\"trainer.kubeflow.org\",\"kind\":\"ClusterTrainingRuntime\"}: ClusterTrainingRuntime.trainer.kubeflow.org \"deepspeed-distributed\" not found: specified clusterTrainingRuntime must be created before the TrainJob is created", Reason: "Forbidden", Details: nil, Code: 403, }, } In [It] at: /workspace/source/src_code/test/e2e/e2e_test.go:148 @ 09/19/26 03:09:19.374 > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:56 @ 09/19/26 03:09:19.329 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:56 @ 09/19/26 03:09:19.337 (8ms) > Enter [It] should create TrainJob with DeepSpeed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:137 @ 09/19/26 03:09:19.337 STEP: Create a TrainJob with deepspeed-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:147 @ 09/19/26 03:09:19.337 END STEP: Create a TrainJob with deepspeed-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:147 @ 09/19/26 03:09:19.374 (37ms) [FAILED] Expected success, but got an error: <*errors.StatusError | 0x2ee407e9ba40>: admission webhook "validator.trainjob.trainer.kubeflow.org" denied the request: spec.RuntimeRef: Invalid value: {"name":"deepspeed-distributed","apiGroup":"trainer.kubeflow.org","kind":"ClusterTrainingRuntime"}: ClusterTrainingRuntime.trainer.kubeflow.org "deepspeed-distributed" not found: specified clusterTrainingRuntime must be created before the TrainJob is created { ErrStatus: { TypeMeta: {Kind: "", APIVersion: ""}, ListMeta: { SelfLink: "", ResourceVersion: "", Continue: "", RemainingItemCount: nil, ShardInfo: nil, }, Status: "Failure", Message: "admission webhook \"validator.trainjob.trainer.kubeflow.org\" denied the request: spec.RuntimeRef: Invalid value: {\"name\":\"deepspeed-distributed\",\"apiGroup\":\"trainer.kubeflow.org\",\"kind\":\"ClusterTrainingRuntime\"}: ClusterTrainingRuntime.trainer.kubeflow.org \"deepspeed-distributed\" not found: specified clusterTrainingRuntime must be created before the TrainJob is created", Reason: "Forbidden", Details: nil, Code: 403, }, } In [It] at: /workspace/source/src_code/test/e2e/e2e_test.go:148 @ 09/19/26 03:09:19.374 < Exit [It] should create TrainJob with DeepSpeed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:137 @ 09/19/26 03:09:19.374 (37ms) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:71 @ 09/19/26 03:09:19.374 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:71 @ 09/19/26 03:09:19.381 (7ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:56 @ 09/19/26 03:09:19.381 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:56 @ 09/19/26 03:09:19.39 (8ms) > Enter [It] should create TrainJob with JAX runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:191 @ 09/19/26 03:09:19.39 STEP: Create a TrainJob with jax-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:197 @ 09/19/26 03:09:19.39 END STEP: Create a TrainJob with jax-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:197 @ 09/19/26 03:09:19.401 (11ms) STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:202 @ 09/19/26 03:09:19.401 END STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:202 @ 09/19/26 03:09:19.663 (262ms) STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:220 @ 09/19/26 03:09:19.663 END STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:220 @ 09/19/26 03:11:13.454 (1m53.791s) < Exit [It] should create TrainJob with JAX runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:191 @ 09/19/26 03:11:13.454 (1m54.064s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:71 @ 09/19/26 03:11:13.454 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:71 @ 09/19/26 03:11:13.461 (7ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:56 @ 09/19/26 03:11:13.461 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:56 @ 09/19/26 03:11:13.468 (7ms) > Enter [It] should create TrainJob with XGBoost runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:249 @ 09/19/26 03:11:13.468 STEP: Create a TrainJob with xgboost-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:255 @ 09/19/26 03:11:13.468 END STEP: Create a TrainJob with xgboost-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:255 @ 09/19/26 03:11:13.508 (39ms) STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:260 @ 09/19/26 03:11:13.508 END STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:260 @ 09/19/26 03:11:13.767 (259ms) STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:278 @ 09/19/26 03:11:13.767 END STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:278 @ 09/19/26 03:11:57.456 (43.689s) < Exit [It] should create TrainJob with XGBoost runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:249 @ 09/19/26 03:11:57.456 (43.987s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:71 @ 09/19/26 03:11:57.456 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:71 @ 09/19/26 03:11:57.46 (4ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:56 @ 09/19/26 03:11:57.46 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:56 @ 09/19/26 03:11:57.469 (9ms) > Enter [It] should create TrainJob with Flux runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:306 @ 09/19/26 03:11:57.469 STEP: Create a TrainJob with flux-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:325 @ 09/19/26 03:11:57.469 END STEP: Create a TrainJob with flux-distributed runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:325 @ 09/19/26 03:11:57.487 (17ms) STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:329 @ 09/19/26 03:11:57.487 END STEP: Wait for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:329 @ 09/19/26 03:11:57.745 (258ms) STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:346 @ 09/19/26 03:11:57.745 END STEP: Wait for TrainJob to be in Succeeded status with all jobs succeeded - /workspace/source/src_code/test/e2e/e2e_test.go:346 @ 09/19/26 03:12:26.695 (28.95s) < Exit [It] should create TrainJob with Flux runtime reference - /workspace/source/src_code/test/e2e/e2e_test.go:306 @ 09/19/26 03:12:26.695 (29.226s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:71 @ 09/19/26 03:12:26.695 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:71 @ 09/19/26 03:12:26.709 (14ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:56 @ 09/19/26 03:12:26.71 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:56 @ 09/19/26 03:12:26.732 (23ms) > Enter [It] should preserve user-provided manager fields - /workspace/source/src_code/test/e2e/e2e_test.go:374 @ 09/19/26 03:12:26.732 STEP: Create a TrainJob with RuntimePatches - /workspace/source/src_code/test/e2e/e2e_test.go:426 @ 09/19/26 03:12:26.732 END STEP: Create a TrainJob with RuntimePatches - /workspace/source/src_code/test/e2e/e2e_test.go:426 @ 09/19/26 03:12:26.776 (44ms) STEP: Verify user-provided Time is preserved for test.io/manager-one and Time is set for kueue.k8s.io/manager - /workspace/source/src_code/test/e2e/e2e_test.go:430 @ 09/19/26 03:12:26.776 END STEP: Verify user-provided Time is preserved for test.io/manager-one and Time is set for kueue.k8s.io/manager - /workspace/source/src_code/test/e2e/e2e_test.go:430 @ 09/19/26 03:12:26.786 (10ms) < Exit [It] should preserve user-provided manager fields - /workspace/source/src_code/test/e2e/e2e_test.go:374 @ 09/19/26 03:12:26.786 (54ms) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:71 @ 09/19/26 03:12:26.786 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:71 @ 09/19/26 03:12:26.808 (22ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:56 @ 09/19/26 03:12:26.808 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:56 @ 09/19/26 03:12:26.831 (23ms) > Enter [It] should update Time only for the changed patch on update - /workspace/source/src_code/test/e2e/e2e_test.go:444 @ 09/19/26 03:12:26.831 STEP: Create a suspended TrainJob with two RuntimePatches - /workspace/source/src_code/test/e2e/e2e_test.go:498 @ 09/19/26 03:12:26.831 END STEP: Create a suspended TrainJob with two RuntimePatches - /workspace/source/src_code/test/e2e/e2e_test.go:498 @ 09/19/26 03:12:26.848 (17ms) STEP: Record the original Time values and verify user-provided Time is preserved - /workspace/source/src_code/test/e2e/e2e_test.go:503 @ 09/19/26 03:12:26.848 END STEP: Record the original Time values and verify user-provided Time is preserved - /workspace/source/src_code/test/e2e/e2e_test.go:503 @ 09/19/26 03:12:26.853 (5ms) STEP: Update the second RuntimePatch and add two new patches - /workspace/source/src_code/test/e2e/e2e_test.go:520 @ 09/19/26 03:12:27.854 END STEP: Update the second RuntimePatch and add two new patches - /workspace/source/src_code/test/e2e/e2e_test.go:520 @ 09/19/26 03:12:27.884 (31ms) STEP: Verify Time behaviour for all patches after update - /workspace/source/src_code/test/e2e/e2e_test.go:539 @ 09/19/26 03:12:27.884 END STEP: Verify Time behaviour for all patches after update - /workspace/source/src_code/test/e2e/e2e_test.go:539 @ 09/19/26 03:12:27.888 (3ms) < Exit [It] should update Time only for the changed patch on update - /workspace/source/src_code/test/e2e/e2e_test.go:444 @ 09/19/26 03:12:27.888 (1.057s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:71 @ 09/19/26 03:12:27.888 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:71 @ 09/19/26 03:12:27.892 (4ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:56 @ 09/19/26 03:12:27.892 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:56 @ 09/19/26 03:12:27.907 (15ms) > Enter [It] should not be reconciled by the built-in controller - /workspace/source/src_code/test/e2e/e2e_test.go:561 @ 09/19/26 03:12:27.907 STEP: Create a TrainJob managed by MultiKueue - /workspace/source/src_code/test/e2e/e2e_test.go:568 @ 09/19/26 03:12:27.907 END STEP: Create a TrainJob managed by MultiKueue - /workspace/source/src_code/test/e2e/e2e_test.go:568 @ 09/19/26 03:12:27.921 (14ms) STEP: Ensuring the built-in controller neither creates a JobSet nor sets any status - /workspace/source/src_code/test/e2e/e2e_test.go:572 @ 09/19/26 03:12:27.921 END STEP: Ensuring the built-in controller neither creates a JobSet nor sets any status - /workspace/source/src_code/test/e2e/e2e_test.go:572 @ 09/19/26 03:12:42.921 (15s) < Exit [It] should not be reconciled by the built-in controller - /workspace/source/src_code/test/e2e/e2e_test.go:561 @ 09/19/26 03:12:42.921 (15.015s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:71 @ 09/19/26 03:12:42.921 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:71 @ 09/19/26 03:12:42.928 (7ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:56 @ 09/19/26 03:12:42.928 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:56 @ 09/19/26 03:12:42.935 (7ms) > Enter [It] should fail the TrainJob with DeadlineExceeded when active timeout expires - /workspace/source/src_code/test/e2e/e2e_test.go:584 @ 09/19/26 03:12:42.935 STEP: Waiting for TrainJob to fail due to deadline - /workspace/source/src_code/test/e2e/e2e_test.go:599 @ 09/19/26 03:12:42.949 STEP: Ensuring the underlying JobSet is deleted - /workspace/source/src_code/test/e2e/e2e_test.go:606 @ 09/19/26 03:12:52.098 < Exit [It] should fail the TrainJob with DeadlineExceeded when active timeout expires - /workspace/source/src_code/test/e2e/e2e_test.go:584 @ 09/19/26 03:12:52.102 (9.167s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:71 @ 09/19/26 03:12:52.102 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:71 @ 09/19/26 03:12:52.108 (6ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:56 @ 09/19/26 03:12:52.108 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:56 @ 09/19/26 03:12:52.118 (10ms) > Enter [It] TrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:837 @ 09/19/26 03:12:52.118 STEP: Creating a TrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:729 @ 09/19/26 03:12:52.118 END STEP: Creating a TrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:729 @ 09/19/26 03:12:52.146 (28ms) STEP: Creating a TrainJob that references the TrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:742 @ 09/19/26 03:12:52.146 END STEP: Creating a TrainJob that references the TrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:742 @ 09/19/26 03:12:52.158 (12ms) STEP: Waiting for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:746 @ 09/19/26 03:12:52.159 END STEP: Waiting for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:746 @ 09/19/26 03:12:52.415 (256ms) STEP: Pausing the TrainJob - /workspace/source/src_code/test/e2e/e2e_test.go:757 @ 09/19/26 03:12:52.415 END STEP: Pausing the TrainJob - /workspace/source/src_code/test/e2e/e2e_test.go:757 @ 09/19/26 03:12:52.432 (17ms) STEP: Waiting for TrainJob to be suspended - /workspace/source/src_code/test/e2e/e2e_test.go:766 @ 09/19/26 03:12:52.432 END STEP: Waiting for TrainJob to be suspended - /workspace/source/src_code/test/e2e/e2e_test.go:766 @ 09/19/26 03:12:52.689 (257ms) STEP: Updating the runtime with a new container image - /workspace/source/src_code/test/e2e/e2e_test.go:779 @ 09/19/26 03:12:52.689 END STEP: Updating the runtime with a new container image - /workspace/source/src_code/test/e2e/e2e_test.go:779 @ 09/19/26 03:12:52.701 (13ms) STEP: Restarting the TrainJob by unpausing it - /workspace/source/src_code/test/e2e/e2e_test.go:800 @ 09/19/26 03:12:52.701 END STEP: Restarting the TrainJob by unpausing it - /workspace/source/src_code/test/e2e/e2e_test.go:800 @ 09/19/26 03:12:52.717 (16ms) STEP: Verifying the TrainJob jobs become active after resuming - /workspace/source/src_code/test/e2e/e2e_test.go:811 @ 09/19/26 03:12:52.717 END STEP: Verifying the TrainJob jobs become active after resuming - /workspace/source/src_code/test/e2e/e2e_test.go:811 @ 09/19/26 03:12:52.974 (256ms) STEP: Verifying the Runtime snapshot is garbage collected - /workspace/source/src_code/test/e2e/e2e_test.go:823 @ 09/19/26 03:12:52.974 END STEP: Verifying the Runtime snapshot is garbage collected - /workspace/source/src_code/test/e2e/e2e_test.go:823 @ 09/19/26 03:12:53.237 (263ms) < Exit [It] TrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:837 @ 09/19/26 03:12:53.237 (1.118s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:71 @ 09/19/26 03:12:53.237 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:71 @ 09/19/26 03:12:53.241 (5ms) > Enter [DeferCleanup (Each)] a suspended TrainJob should use the original runtime configuration and not pick up the new configuration - /workspace/source/src_code/test/e2e/e2e_test.go:733 @ 09/19/26 03:12:53.241 < Exit [DeferCleanup (Each)] a suspended TrainJob should use the original runtime configuration and not pick up the new configuration - /workspace/source/src_code/test/e2e/e2e_test.go:733 @ 09/19/26 03:12:53.246 (5ms) > Enter [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:56 @ 09/19/26 03:12:53.246 < Exit [BeforeEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:56 @ 09/19/26 03:12:53.255 (9ms) > Enter [It] ClusterTrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:849 @ 09/19/26 03:12:53.255 STEP: Creating a ClusterTrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:729 @ 09/19/26 03:12:53.255 END STEP: Creating a ClusterTrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:729 @ 09/19/26 03:12:53.279 (24ms) STEP: Creating a TrainJob that references the ClusterTrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:742 @ 09/19/26 03:12:53.279 END STEP: Creating a TrainJob that references the ClusterTrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:742 @ 09/19/26 03:12:53.292 (13ms) STEP: Waiting for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:746 @ 09/19/26 03:12:53.292 END STEP: Waiting for TrainJob jobs to become active - /workspace/source/src_code/test/e2e/e2e_test.go:746 @ 09/19/26 03:12:53.548 (256ms) STEP: Pausing the TrainJob - /workspace/source/src_code/test/e2e/e2e_test.go:757 @ 09/19/26 03:12:53.548 END STEP: Pausing the TrainJob - /workspace/source/src_code/test/e2e/e2e_test.go:757 @ 09/19/26 03:12:53.562 (14ms) STEP: Waiting for TrainJob to be suspended - /workspace/source/src_code/test/e2e/e2e_test.go:766 @ 09/19/26 03:12:53.562 END STEP: Waiting for TrainJob to be suspended - /workspace/source/src_code/test/e2e/e2e_test.go:766 @ 09/19/26 03:12:53.819 (257ms) STEP: Updating the runtime with a new container image - /workspace/source/src_code/test/e2e/e2e_test.go:779 @ 09/19/26 03:12:53.819 END STEP: Updating the runtime with a new container image - /workspace/source/src_code/test/e2e/e2e_test.go:779 @ 09/19/26 03:12:53.832 (13ms) STEP: Restarting the TrainJob by unpausing it - /workspace/source/src_code/test/e2e/e2e_test.go:800 @ 09/19/26 03:12:53.832 END STEP: Restarting the TrainJob by unpausing it - /workspace/source/src_code/test/e2e/e2e_test.go:800 @ 09/19/26 03:12:53.849 (17ms) STEP: Verifying the TrainJob jobs become active after resuming - /workspace/source/src_code/test/e2e/e2e_test.go:811 @ 09/19/26 03:12:53.849 END STEP: Verifying the TrainJob jobs become active after resuming - /workspace/source/src_code/test/e2e/e2e_test.go:811 @ 09/19/26 03:12:54.106 (257ms) STEP: Verifying the Runtime snapshot is garbage collected - /workspace/source/src_code/test/e2e/e2e_test.go:823 @ 09/19/26 03:12:54.106 END STEP: Verifying the Runtime snapshot is garbage collected - /workspace/source/src_code/test/e2e/e2e_test.go:823 @ 09/19/26 03:12:54.369 (263ms) < Exit [It] ClusterTrainingRuntime - /workspace/source/src_code/test/e2e/e2e_test.go:849 @ 09/19/26 03:12:54.369 (1.114s) > Enter [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:71 @ 09/19/26 03:12:54.369 < Exit [AfterEach] TrainJob e2e - /workspace/source/src_code/test/e2e/e2e_test.go:71 @ 09/19/26 03:12:54.375 (6ms) > Enter [DeferCleanup (Each)] a suspended TrainJob should use the original runtime configuration and not pick up the new configuration - /workspace/source/src_code/test/e2e/e2e_test.go:733 @ 09/19/26 03:12:54.375 < Exit [DeferCleanup (Each)] a suspended TrainJob should use the original runtime configuration and not pick up the new configuration - /workspace/source/src_code/test/e2e/e2e_test.go:733 @ 09/19/26 03:12:54.38 (5ms)